Archives For ESX

After making changes to the service console IP addresses I have found an issue with the APC agent on the ESX Server. When trying to change the configuration to reflect the new UPS IP addresses I got an error saying unable to edit configuration file. On trying to uninstall the RPM I recived an error saying it wasn’t installed, on trying to re-install it I recieved an error sayig it was already installed. To resolve this I ran the following command on the RPM

rpm -ihv –replacepkgs pcns-2.2.1-100.i386.rpm

This forced a reinstall of the RPM and corrected the issue, I am now able to reconfigure the UPS for network shutdown.

Recently after changing the IP address of several ESX hosts I was experiencing a problem with HA on one of them. Looking in the event log I was getting the following error “Configuration of host IP address is inconsistent on host” after a lot of troubleshooting I found the FT_Hosts still had the old IP’s. After updating this file all sprung into life.

I have since found this post by Duncan Epping that would have helped considerably at the time http://www.yellow-bricks.com/2008/06/04/changing-the-ip-address-of-an-esx-host-and-ha/

You need to log onto the SC and run these commands;

esxcfg-vswitch -l (lists all your vswitches and port groups)

Find the portgroup name that has the SC assigned to it (usually port group 0)

esxcfg-vswitch vSwitch0 -p “portgroup name” -v “VLANID” or
esxcfg-vswitch vSwitch0 -p “portgroup name” -v 0 to disable the VLANID

OK all I seem to go on about at the moment is Virtualization Eco Shell and powershell, being a bit of a newbie to Powershell I have been amazed by just what I can do with it and have been eager to learn more. Whilst I have been learning and modifying code I have found on the internet, I have also started using Virtualization Eco Shell since its release.

The Virtualization Eco Shell is now the first tool I open when I get to site, I find it quicker and easier to get the information I need about an enviroment than using the VI / vSphere client.

A few examples of how I have recently used it

Reporting number of CPU’s and RAM in all the VM’s in an enviroment, using the advanced reporting pack that can be downloaded here >> http://thevesi.org/downloads.jspa within seconds I was able to produce a customised report similar to that below for a system with well over a hundred VM’s.

vmsummary2

Upon discovering a large amount of VMs we not time syncing with the host through VMware tools you would usually have to change the setting on each guest. After posting a question on the www.thevesi.org forum Scott has now added this functionality to the Virtualization Eco Shell >> http://thevesi.org/message.jspa?messageID=29595#29595

I am now able to set all my VM’s to sync at the click of a button

synctimewithhost

As for the vDiagrams they are fantastic, having previously used Veeam reporter to draw diagrams for infrastructures the vDiagrams option does it miles better and for free. Although I haven’t used the latest version of Veeams Reporter so this may have improved significantly. I just want the option to be able to diagram my network now as well! Maybe thats asking a bit much for a free product.

vmdiagram

For the latest information keep an eye on www.thevesi.org and also be sure to check out their blog here >> http://blog.thevesi.org/

I found this fantastic article on installing HP agents for ESX for more information visit here >> http://vikashkumarroy.blogspot.com/2009/02/installing-hp-insight-management-agents.html

Fantastic article on VMotion CPU Compatibility Requirements for Intel Processors, taken from >> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991

 

In ESX 3.5 Update 2 and later, VMware recommends using Enhanced VMotion Compatibility (EVC) to eliminate many VMotion CPU compatibility problems. For more information on EVC, see KB 1003212.   To ensure system stability during migration with VMotion, VirtualCenter and vCenter Server require the source and target CPUs to be compatible. For more information on CPU compatibility requirements, see Basic System Administration for your version of VirtualCenter or vCenter Server, available from http://www.vmware.com/support/pubs/.

If the source and target CPUs are incompatible for VMotion, you can:

  • Perform a cold migration (rather than a VMotion migration), thereby removing VMotion CPU requirements as an issue.
  • Remove VMotion compatibility constraints by modifying the default bit-mask used by VirtualCenter or vCenter Server. Note that some modifications discussed in this knowledge base article are neither supported nor recommended by VMware for production environments. In general, masking any CPU features intended for applications (such as SSE3) is not supported for VMotion. Use of EVC is recommended for migrating virtual machines across CPU generations.

To obtain more information about a host system’s CPU, you can use the CPU Identification Utility. VMware provides this as an ISO image file that can be uncompressed and used to create a bootable CD-ROM that provides CPU information about a host, even before an operating system or ESX is installed. The latest version of this tool can be found on the VMware downloads page at http://vmware.com/download/shared_utilities.html.

This knowledge base article discusses VMotion compatibility constraints for Intel CPUs. For detailed information about how to apply the masks discussed in this article, see knowledge base article 1993, “Migrations with VMotion Prevented Due to CPU Mismatch—How to Override Masks.” For information on AMD CPUs, see KB 1992.  

VMotion Compatibility Groups for Intel Processors

To guarantee successful migrations with VMotion, VMware has defined several compatibility groups based on processor family (Pentium 4, Core) and features introduced within those families.

By default, VirtualCenter and vCenter Server support VMotion migrations within each CPU compatibility group. For example, migration within group A is allowed, but migration from group A to group B or from group B to group C is not, as shown in the tables.

Intel Pentium 4 CPUs

VMotion CPU Compatibility Group CPU Details  ESX Server 3.x and ESX 4.x  ESX Server 2.x  
                Group A Without SSE3, without XD (eXecute Disable).Models include:

  • Intel P4s prior to Model 3, Stepping 1.
    For example, Willamette and Northwood.
  • Xeon and Xeon MP CPUs prior to Nocona, Q3 2004. For example, Foster, Prestonia, and Gallatin.
      For A <-> B VMotion, apply SSE3 mask (not supported).

 

 

 

 

 

 

For A <-> B VMotion, apply SSE3 mask (not supported).

 

 
          Group B
(Group B and C are the same for VC 1.x) 
With SSE3, without XD. Models include:P4s from Model 3, Stepping 1 to Model 4, Stepping 1.
For example, Prescott, or numbered 5×0, 5×5, 5×9.
 
 

          For B <-> C VMotion, apply NX mask (supported).

 

 
                Group C
(Group B and C are the same for VC 1.x) 
With SSE3 and XD. Models include:P4s after Model 4, Stepping 1 onward and Xeon and Xeon MP with 64-bit (EM64T) enabled.
For example, Irwindale, Cranford, Dempsey, Tulsa, or numbered 50xx, 70xx, or 71xx.
 

Intel Core CPUs

VMotion CPU Compatibility Group CPU Details ESX 4.x, ESX Server 3.x, and ESX Server 2.x
Group A Without SSSE3, SSE4.1, or SSE4.2. Models include: Dual-core Xeon LV based on Intel Core microarchitecture. For example, Sossaman. For A<->B VMotion, apply SSSE3 mask (not supported).
Group B With SSSE3. Without SSE4.1 or SSE 4.2. Models include: Intel Xeon CPUs based on the Intel Core microarchitecture. For example, Intel Xeon 30xx, 32xx, 51xx, 53xx, 72xx, or 73xx.
For B<->C VMotion, apply SSE4.1 mask (not supported). 
Group C With SSSE 3 and SSE4.1. Without SSE4.2. Models include: Intel Xeon CPUs based on 45nm Intel Core microarchitecture. For example, Intel Xeon 31xx, 33xx, 52xx, 54xx, or 74xx.
For C<->D VMotion, apply SSE4.2 mask (not supported). 
Group D With SSSE3, SSE4.1, and SSE4.2. Models include: Intel Xeon CPUs based on Intel Nehalem microarchitecture (Core i7). For example, Intel Xeon 55xx.

Applying the Masks

For information about how to apply masks referenced in the tables, see KB 1993 under the section titled “Modifying the Default Mask”.   Warning: For production environments, VMware neither supports nor recommends modifying CPU masks for SSE3, SSSE3, SSE4.1, or SSE4.2, because of the risk of failure in applications or the guest operating system after migration.  

VMotion Between Single-Core and Multi-Core Processors

Migrations between single-core and multi-core Intel processors are supported, as long as the source and target CPUs have compatible CPU features (or the features are masked) as outlined in the tables above.   

In ESX 3.5 Update 2 and later, VMware recommends using Enhanced VMotion Compatibility (EVC) to eliminate many VMotion CPU compatibility problems. For more information on EVC, see KB 1003212.
 
To ensure system stability during migration with VMotion, VirtualCenter and vCenter Server require the source and target CPUs to be compatible. For more information on CPU compatibility requirements, see Basic System Administration for your version of VirtualCenter or vCenter Server, available from http://www.vmware.com/support/pubs/.

If the source and target CPUs are incompatible for VMotion, you can:

  • Perform a cold migration (rather than a VMotion migration), thereby removing VMotion CPU requirements as an issue.
  • Remove VMotion compatibility constraints by modifying the default bit-mask used by VirtualCenter or vCenter Server. Note that some modifications discussed in this knowledge base article are neither supported nor recommended by VMware for production environments. In general, masking any CPU features intended for applications (such as SSE3) is not supported for VMotion. Use of EVC is recommended for migrating virtual machines across CPU generations.

To obtain more information about a host system’s CPU, you can use the CPU Identification Utility. VMware provides this as an ISO image file that can be uncompressed and used to create a bootable CD-ROM that provides CPU information about a host, even before an operating system or ESX is installed. The latest version of this tool can be found on the VMware downloads page at http://vmware.com/download/shared_utilities.html.

This knowledge base article discusses VMotion compatibility constraints for Intel CPUs. For detailed information about how to apply the masks discussed in this article, see knowledge base article 1993, “Migrations with VMotion Prevented Due to CPU Mismatch—How to Override Masks.” For information on AMD CPUs, see KB 1992.
 

VMotion Compatibility Groups for Intel Processors

To guarantee successful migrations with VMotion, VMware has defined several compatibility groups based on processor family (Pentium 4, Core) and features introduced within those families.

By default, VirtualCenter and vCenter Server support VMotion migrations within each CPU compatibility group. For example, migration within group A is allowed, but migration from group A to group B or from group B to group C is not, as shown in the tables.

Intel Pentium 4 CPUs

VMotion CPU Compatibility Group CPU Details  ESX Server 3.x and ESX 4.x  ESX Server 2.x  
 
 
 
 
 
 
 
 
Group A
Without SSE3, without XD (eXecute Disable).

Models include:

  • Intel P4s prior to Model 3, Stepping 1.
    For example, Willamette and Northwood.
  • Xeon and Xeon MP CPUs prior to Nocona, Q3 2004. For example, Foster, Prestonia, and Gallatin.
 
 
 
For A <-> B VMotion, apply SSE3 mask (not supported).

 

 

 

 

 

 

For A <-> B VMotion, apply SSE3 mask (not supported).

 

 
 
 
 
 
 
Group B
(Group B and C are the same for VC 1.x)

 

With SSE3, without XD.
Models include:

P4s from Model 3, Stepping 1 to Model 4, Stepping 1.
For example, Prescott, or numbered 5×0, 5×5, 5×9.

 
 

 
 
 
 
 
For B <-> C VMotion, apply NX mask (supported).

 

 
 
 
 
 
 
 
 
 
Group C
(Group B and C are the same for VC 1.x)

 

With SSE3 and XD.
Models include:

P4s after Model 4, Stepping 1 onward and Xeon and Xeon MP with 64-bit (EM64T) enabled.
For example, Irwindale, Cranford, Dempsey, Tulsa, or numbered 50xx, 70xx, or 71xx.

 

Intel Core CPUs

VMotion CPU Compatibility Group
CPU Details
ESX 4.x, ESX Server 3.x, and ESX Server 2.x
Group A
Without SSSE3, SSE4.1, or SSE4.2.
Models include:
Dual-core Xeon LV based on Intel Core microarchitecture.
For example, Sossaman.
For A<->B VMotion, apply SSSE3 mask (not supported).
Group B
With SSSE3. Without SSE4.1 or SSE 4.2.
Models include:
Intel Xeon CPUs based on the Intel Core microarchitecture. For example, Intel Xeon 30xx, 32xx, 51xx, 53xx, 72xx, or 73xx.
For B<->C VMotion, apply SSE4.1 mask (not supported). 
Group C
With SSSE 3 and SSE4.1. Without SSE4.2.
Models include:
Intel Xeon CPUs based on 45nm Intel Core microarchitecture. For example, Intel Xeon 31xx, 33xx, 52xx, 54xx, or 74xx.
For C<->D VMotion, apply SSE4.2 mask (not supported). 
Group D
With SSSE3, SSE4.1, and SSE4.2.
Models include:
Intel Xeon CPUs based on Intel Nehalem microarchitecture (Core i7). For example, Intel Xeon 55xx.

Applying the Masks

For information about how to apply masks referenced in the tables, see KB 1993 under the section titled “Modifying the Default Mask”.
 
Warning: For production environments, VMware neither supports nor recommends modifying CPU masks for SSE3, SSSE3, SSE4.1, or SSE4.2, because of the risk of failure in applications or the guest operating system after migration.
 

VMotion Between Single-Core and Multi-Core Processors

Migrations between single-core and multi-core Intel processors are supported, as long as the source and target CPUs have compatible CPU features (or the features are masked) as outlined in the tables above.  

For anyone looking to learn more about PowerCLI or VI Toolkit I can highly recommend watching the PowerCLI – What is new in PowerCLI by Carter Shanklin on VMware’s coffee talk webinars page here >>http://is.gd/Op6J 

For me this session was fantastic it really opened my eyes to some of the more advanced methods of getting data using PowerCLI, there are a number of commands that will assist you in day to day troubleshooting techniques.

I blogged a while ago about downloading the Virtualisation Eco Shell (VESI), since then I haven’t stopped using it. It now forms one of my tools that is opened as soon as I am dealing with VMware. VESI puts a graphical user interface over the top of the VI Toolkit / PowerCLI and powershell. Allowing you to run queries and get information out of a environment in a matter of seconds. Below is a short list of some of the ways I have used it recently. I plan on doing a more detailed blog post shortly depicting how usefull this free tool is. Big thanks must go to Scott Herold who runs the project, he has been extremly helpful  and accommodating with assistance and feature requests.

  • Quickly shutdown all VM’s when doing SAN maintainance
  • Documentation / Infrastructure Diagrams
  • Checking host log files
  • Checking for snapshots
  • Using the script editior to write, test and debug powershell scripts
  • Checking Windows service status / restarting / starting services
  • Checking Windows event logs for VMs

Plus lots more.

For more information please visit the website www.thevesi.org

As I have mainly installed ESX servers, today I was posed with a situation where I needed to restart the management agents on an ESXi server. Immediatly I was thinking I would need to access the emergency service console etc but I found the convienent restart management agents option in the F2 menu.

To access do the following

1. From the console screen (use ilo / drac connection to connect if needed) press F2

2. Login

3. Now press F11 to restart the agents

You can now logout again, this is a usual one to resolve a host that is appearing as disconnected and you are unable to manage it.

VMware Snapshots

May 22, 2009 — 2 Comments

After another customer recently had an issue with old VMware snapshots, I thought I would put together some pointers regarding VMware snapshots on production servers and some items I discussed with the VMware engineer at the time.

  • Snapshots are not a backup and should not be treated as such, snapshots should be used as a moment in time recovery point whilst undertaking installations of updates and new software etc on a server. Once you are happy remove the snapshot immediately.
  • Only keep snapshots live for the shortest amount of time possible.
  • As a rule of thumb ensure your snapshots are deleted at the latest within 24 hours, if you have a requirement to keep them longer consider taking a backup instead.
  • Don’t remove snapshots during your servers busiest times
  • If you have a very old snapshot consider either cloning the VM to a new VM (This will consolidate the snapshots and keep the original for fail back) or turning off the VM and removing the snapshot, this will mean there is no change happening to the delta whilst it is being consolidated.
  • When removing large snapshots from the VI client it will timeout after around 15 minutes.  This doesn’t mean it has failed! Be paitent check via the service console the progress by looking at the datestamp on the VMDK.
  • Check regularly to ensure there are no outstanding snapshots, a tool such as the Virtualisation Eco Shell will help you with this.
  • If you wish to check for snapshots from the service console use the following command, this will show you the location and size of the delta files.  

 find /vmfs/volumes/ -name “*delta*” -type f  -print0 | xargs -0 du –human-readable –total

Please feel free to comment other suggestions, I will add to this other time.

Cheers

Barry