Core Hardware Options for ESX / ESXi Server

$vmkvsitools hwinfo -i
So, what else “vmkvsitools” can do?. Although, I haven’t finished yet exploring all the parameters, you can try some of the commands as I listed below :-

Usage: ‘vmkvsitools CMD option’

CMD: amldump, bootOption, hwclock, hwinfo, lsof, lspci, pci-info, pidof, ps, vdf, vdu, vmksystemswap, vmware
eg1. “$vmkvsitools CMD -h” -> get help
eg2. “$vmkvsitools hwinfo -p” -> print all pci device info present
eg3. “$vmkvsitools hwinfo -h” -> Print usage
eg4. “$vmkvsitools lspci -p” -> print details of pci device
eg5. “$vmkvsitools vmware -v” vmkernel version
eg6. “$vmkvsitools vmware -l” Print release level
eg7. “vmkvsitools hwclock -d 10/07/2010 -t 00:33:00″ Set date & time
eg6. “vmkvsitools ps -c” Print process ID (PID) with verbose command aka ps auxwww


Well, just give a try because too much to explain here. Remember “$vmkvsitools CMD -h” for help.

How to connect the below window from command line ?




~#cd /usr/sbin...............>

/sbin # dcui/
What is amldump?

~ # amldump

Writing file DSDT.aml

Writing file FACS.aml

Writing file FACP.aml

Writing file SSDT.aml

Writing file MSDM.aml

Writing file HPET.aml

Writing file MCFG.aml

Writing file TAMG.aml

Writing file APIC.aml

~ #

Regulary which is used for ESX/ESXI Server in Core level for background process of Operating system

techsupport.sh

vmkdump_extract

amldump

ntpd

vmkerrcodeapply-host-profiles

vmkeventd
applyHostProfile

vmkfstools
esxtop 

vmkiscsi-tool
auto-backup.sh

esxupdate

partedUtil 

vmkiscsid
backup.sh

ethtool

pidof

chkconfig

firmwareConfig.sh

vmkmkdev

dcbd

ft-stats

randomSeed

vmkperfdcui

generate-certificates 
vmkramdisk
df 

hbrfilterctl

scantools

vmksystemswap
dmesg

sensord

vmtar
doat

hwclock

vmware-autostart.shhwinfo 

sfcbd

vmware-usbarbitratorshutdown.sh 
esxcfg-dumppart 

smbiosDump

vscsiStatsesxcfg-fcoe watchdog.sh

logchannellogger

storageRM

lspci 

tmpwatch.py
lsusb

traceroute
uptime
memstats 

vm-support
net-dvs

vmdumper
net-fence

vmfs-support
net-lbt

vmkbacktrace
netlogond

vmkchdev
Session.py 

vmkdevmgr
ntp-keygen 

localcli 

statedumper

vmkload_mod
bootOption

fdisk 

vmkmicrocode

Run Hardware Diagnostic tests

Most servers are shipped with a hardware diagnostics CD, although other hardware vendors may choose to install a hidden utility partition located on your hard drive.

Note: If you are not experienced with computers or have any concerns, please contact your hardware vendor.

You can diagnose hardware related problems on your server by booting from the diagnostic CD or choosing Diagnostics from the boot device list.

These diagnostic tools allow you to:

• Check the hardware configuration and verify that it is functioning correctly.
• Test individual hardware components.
• Diagnose hardware-related problems.
• Obtain a complete hardware configuration.

When testing, if a component failure is detected, make note of any error code(s) and contact the hardware vendor.

Check your memory

Note: This process requires downtime on your ESX/ESXi host for up to 48 hours. In most cases, contacting your hardware vendor for a diagnostic utility as mentioned above should be sufficient in testing your hardware. VMware does not endorse or recommend any particular third party utility.  However, there are third party options available to test your memory.

To test your memory:

1. Download memtest86+ from http://www.memtest.org/.
2. Extract the ISO image from the .gz or .zip archive.
3. Burn the image to CD.
4. Boot your ESX/ESXi host from the CD.
5. The memtest goes through each memory bank and checks for errors.
   
   Note: If memtest86+ does not run on your hardware, contact your vendor for their memory test utility. 

Ensure your server conforms to Non-Uniform Memory Access (NUMA) rules and regulations

Notes:

• If you are not experienced with computers or have any concerns, please contact your hardware vendor.
• Problems related to NUMA usually occur following a RAM upgrade or after an ESX/ESXi Server host installation.

You might see an error such as the following:

The BIOS reports that NUMA node 1 has no memory. This problem is either caused by a bad BIOS or a very unbalanced distribution of memory modules. 

NUMA is a system where each processor has separate memory. The separate memory helps to avoid a performance hit when several processors attempt to address the same memory.

The main requirement is that a similar amount of memory is installed beside each processor. If the amount of memory installed beside each processor is not not similar, it is unbalanced and you might experience performance problems.For more information, see ESX Server Memory Management on Systems with AMD Opteron Processors (1570).

More information on NUMA is also available in the Resource Management Guide.

Run the VMware CPU Identification Utility

To ensure that your CPU(s) are being detected as expected you can use the VMware CPU Identification Utility. You can download the utility at VMware Shared Utilities. This tool helps you ensure that the ESX host is detecting and reporting your CPU(s) correctly.

When the the VMware CPU Identification Utility has been downloaded, the cpuid.iso image can be used to create a bootable CD that aids in processor and feature identification. The tool displays Family/Model/Stepping information for the CPUs detected, and hexadecimal values for the CPU registers that identify specific CPU features. The hexadecimal register values are then interpreted to indicate whether the CPUs support features like, 64bit, SSE3, and nX/xD.

The following is sample output:

Reporting CPUID for 2 logical CPUs...

All CPUs are identical

Family: 0f Model: 04 Stepping: 1

ID1ECX ID1EDX ID81ECX ID81EDX
0x0000641d 0xbfebfbff 0000000000 0x20100000Vendor : Intel
Processor Cores : 1
Brand String : " Intel(R) Xeon(TM) CPU 2.80GHz"
SSE Support : SSE1, SSE2, SSE3
Supports NX / XD : Yes
Supports CMPXCHG16B : Yes
Hyperthreading : Yes
Supports 64-bit Longmode : Yes
Supports 64-bit VMware : No

Additional Information

In addition make sure that you are meeting the minimum system requirements for your ESX/ESXi.  For more information see Minimum requirements for installing ESX/ESXi (1003661).For more information about decoding machine check exceptions, see  Decoding Machine Check Exception (MCE) output after a purple screen error (1005184)

Hardware & VCLI Commands

There might be a few reasons that you would need to do this. But if you need to locate the Serial Number of server or Service Tag of your Dell server you can do this from the service console command line.  In the past I have needed this to schedule service and also to confirm the identity of a server for the Vendor that was on site. In case you do not have a database to reference or maybe someone mistyped the digits you can always fall back to this method.

[root@host name]#  /usr/sbin/dmidecode |grep -A4 “System Information”

I have grabbed a list of the new commands added to vCLI 4.1, these command will help narrow the gap that had existed between what you could run on the ESX console (COS) and what you could do via the vCLI with an ESXi host. Notice the part at the end where it lists some of the commands that cannot be executed against a vCenter server for a host in lock down mode.

• vicfg-hostops – Allows you to examine, stop, and reboot hosts and to instruct hosts to enter and exit maintenance mode.
• vicfg-authconfig – Allows you to add an ESX/ESXi host to an Active Directory domain, remove the host, and list Active Directory domain information.
• vicfg-ipsec – Supports IPsec setup.

vSphere CLI 4.1 also includes the following new functionality:

• The following options have been added to esxcli:
  • esxcli swiscsi session – Manage iSCSI sessions.
  • esxcli swiscsi nic – Manage iSCSI network interfaces.
  • esxcli swiscsi vmknic – List VMkernel network interfaces available for binding to particular iSCSI adapter.
  • esxcli swiscsi vmnic – List available uplink adapters for use with a specified iSCSI adapter.
  • esxcli vaai device – Display information about devices claimed by the VMware VAAI (vStorage APIs for Array Integration) Filter Plugin.
  • esxcli corestorage – List devices or plugins. Used in conjunction with hardware acceleration.
  • esxcli network – List active connections or list active ARP table entries.
  • esxcli vms – List and forcibly stop virtual machines that do not respond to normal stop operations.
• Some of the parity issues between vSphere CLI and the ESX service console have been resolved.
• You can now run vCLI commands using SSPI (--passthroughauth) against both vCenter Server and ESX/ESXi systems.
• Lockdown mode allows vSphere administrators to block direct access to ESXi systems. With lockdown mode enabled, all operations must go through a vCenter Server system. The following commands cannot run against vCenter Server systems and can therefore not be used in lockdown mode:
  • vicfg-snmp
  • vifs
  • vicfg-user
  • vicfg-cfgbackup
  • vihostupdate
  • vmkfstools
  • esxcli
  • vicfg-ipsec
If you want to run these commands against an ESXi system, turn off lockdown mode using the vSphere Client.

Hardware Issues for ESX / ESXI Server

If you are using ESX 4.0 (Classic) then try running following command on your esx server  :-   #dmidecode > out.txt or if you have ESXi 4.0 then try this command #smbiosDump > out.txt

In dimidecode and smbiosdump output,  you will know in which slot memory is installed and also size of the memory in each slot. 

I am not sure how to check failed memory but in demidecode and smbiosdump output, Their is property like Error Info.. Check that property in your scenario.

How to Check the pci Information in ESX/ESXI Servers
Use the VMware provided tool “vmkvsitools“. Go to the console and issue the command “vmkvsitools lspci“. This command returns a list of all PCI devices in your system, equal to just running the “lspci” command, but adds the VMware device name at the end of each line

Upgrading NIC drivers for VMware vSphere (ESX/ESXi) 4.x Hosts

Contents:

• Overview of Upgrading NIC drivers in vSphere 4
• Steps to Update NIC drivers in vSphere 4
• Conclusion

Overview of Upgrading NIC drivers in vSphere
With our ever growing technology infrastructure we get to play the update and upgrade games so everything can talk to everything else. Even virtualization does not free us from the fun exercise of upgrading drivers and firmware levels. If, for whatever reason, you find yourself asking "how do I upgrade the NIC drivers on a vSphere host?" The steps below should provide enough information to get you through the process.

Please note that these steps were performed on a Windows 2008 R2 Enterprise server and were ran against a VMware ESX 4.1.0 build 260247 vSphere host using the VMware vSphere CLI 4.1.0 build 254719 tool on December 8th, 2010. These steps should work for any version of vSphere 4.x and vCLI 4.x but read the documentation on the 'vihostupdate' command from VMware's website first to verify the process has not changed. If you are running the vSphere CLI from a *nix host please read the documentation to verify the steps outlined will work for you before attempting them.
The documentation is located here: http://www.vmware.com/support/developer/vcli/.

A reboot of the VMware vSphere ESX / ESXi host is required before the changes will take effect. The reboot will not occur automatically and should be performed at step 13.

These instructions are not meant to be exhaustive and may not be appropriate for your environment. Always check with your vendors for the appropriate steps to manage your infrastructure.

Steps to Update NIC drivers in vSphere 4

1. Install the vSphere Command Line Interface (vSphere CLI) on a machine that has SSH access to your vSphere ESX/ESXi host's service console.
   The Installer is located here: http://www.vmware.com/download/download.do?downloadGroup=VCLI41
   VMware recommends installing the vSphere CLI on your vSphere vCenter Server as it typically has access to your vSphere hosts but, it does not have to be on a vCenter Server.
   
2. Verify your version of ESX and/or ESXi.
   For an ESX host, SSH to the service console and perform the following:
   
   [root@MYHOST1 ~]# vmware -v
   VMware ESX 4.1.0 build-260247
   
   For an ESXi host, connect to the console with a KVM, lights out manager, or other management tool and view the home screen:
   
   
   
3. Verify the NIC type you are running and the driver name.
   From a vCenter client, change your view to Hosts and Clusters.
   
   
   
   Expand the cluster the host is in and select the vSphere Host you want to upgrade. Click on the configuration tab, then networking, and click properties next to a NIC (in Virtual Switch or Distributed Switch view). The name of your network adapter is located in the Network Adapters tab and the Adapter Details pane. The name of the driver is in the driver field. From this example the name of the NIC is Broadcom NetXtreme II 57711E and the driver is bnx2x.
   
   
   
4. Place your host in maintenance mode through the vSphere Client and wait for all virtual machines to be migrated off the host before continuing.
   
   
5. Verify the current running version of your drivers.
   For an ESX host, SSH to the service console and perform the below command. Replace "bnx2x" with your driver name from step 3 (above). The current version is the driver with the text "installed" in the second field. From this example the "bnx2x_400.1.54.1" driver is the current version.
   
   [root@MYHOST1 ~]# esxupdate query --vib-view | grep bnx2x
   rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-1.0.7.193498@x86_64     retired
   rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-2vmw.1.9.208167@x86_64     retired
   rpm_vmware-esx-drivers-net-bnx2x_400.1.54.1.v41.1-1vmw.0.0.260247@x86_64     installed
   rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-2vmw.2.17.261974@x86_64     retired
   
   
6. Download the appropriate driver for your host's NIC and vSphere ESX / ESXi version fromhttp://downloads.vmware.com/d/info/datacenter_downloads/vmware_vsphere_4/4#drivers_tools. You may have to expand the "Driver CDs" menu to view all downloads.
   
7. Copy the data from the CD/offline-bundle/ folder to the machine that you installed the vSphere CLI on.
   Depending on your network adapter and the types of drivers available, you may have 1 to many zip files in the offline-bundle folder. Your drivers may be a different name and version than what is shown below.
   
   
   
8. Run the vSphere CLI and change to the directory that you saved the offline-bundle zip files to using the "cd" command.
   
9. Verify the offline-bundle package you downloaded is valid for your system. Make sure to change the file specified after "--bundle" to the zip file that corresponds to the driver you discovered from step 3 (above). (Time saving tip: you should be able to tab complete the driver name.)
   
   C:\drivers\offline-bundle>vihostupdate.pl --server 172.16.1.2 --scan --bundle BCM-bnx2x-1.60.50.v41.2-offline_bundle-325733.zip
   Enter username:
   Enter password:
   The bulletins which apply to but are not yet installed on this ESX host are listed.
   
   ---------Bulletin ID-------------------------Summary-----------------
   BCM-bnx2x-1.60.50.v41.2     bnx2x: net driver for VMware ESX
   
   If you receive a message stating there are no bulletins which apply to your system then you have either grabbed the wrong driver cd or your system is already up to date.
   
10. Install the offline-bundles which apply to your system. It is possible to define the username and password on the command line or through a configuration file. Read the vSphere Command-line reference document's section titled vSphere CLI Connection Options for more information.
    
    C:\drivers\offline-bundle>vihostupdate.pl --server 172.16.1.2 --install --bundle BCM-bnx2x-1.60.50.v41.2-offline_bundle-325733.zip
    Enter username:
    Enter password:
    Please wait patch installation is in progress ...
    The update completed successfully, but the system needs to be rebooted for the changes to be effective.
    
    The driver install typically takes about a minute but could be faster or slower depending on your system configuration.
    
11. Repeat steps 9 and 10 as necessary for any remaining network drivers.
    
12. Verify the drivers installed correctly.
    For an ESX host, SSH to the service console and perform the below command. Be sure to replace "bnx2x" with your driver name from step 3 (above). You should see a driver (or drivers) with a status of "pending,installed". In this example the "bnx2x_400.1.60.50" driver was installed over the old "bnx2x_400.1.54.1" driver.
    
    [root@MYHOST1 ~]# esxupdate query --vib-view | grep bnx2x
    rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-1.0.7.193498@x86_64     retired
    rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-2vmw.1.9.208167@x86_64     retired
    rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-2vmw.2.17.261974@x86_64     retired
    cross_vmware-esx-drivers-net-bnx2x_400.1.60.50.v41.2-1vmw.0.0.00000     pending,installed
    rpm_vmware-esx-drivers-net-bnx2x_400.1.54.1.v41.1-1vmw.0.0.260247@x86_64     retired
    
    If the new driver version does not appear it is possible the driver install failed from step 10 or the wrong driver was chosen for installation.
    
13. Reboot the newly upgraded host through the vSphere Client.
    
14. When the host is online, verify the new drivers are installed and running.
    For an ESX host, SSH to the service console and perform the below command. Be sure to replace "bnx2x" with your driver name from step 3 (above). You should see a driver (or drivers) with a status of "installed". In this example the "bnx2x_400.1.60.50" driver was installed successfully and is running after the reboot.
    
    [root@USSLTCHER0049 ~]# esxupdate query --vib-view | grep bnx2x
    rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-1.0.7.193498@x86_64     retired
    rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-2vmw.1.9.208167@x86_64     retired
    rpm_vmware-esx-drivers-net-bnx2x_400.1.45.20-2vmw.2.17.261974@x86_64     retired
    cross_vmware-esx-drivers-net-bnx2x_400.1.60.50.v41.2-1vmw.0.0.00000     installed
    rpm_vmware-esx-drivers-net-bnx2x_400.1.54.1.v41.1-1vmw.0.0.260247@x86_64     retired

Conclusion
Following the above process you should be able to successfully upgrade the NIC drivers for your vSphere 4 ESX / ESXi hosts. Always be sure to read the release details of your drivers before download and installing them. If you have any problems updating your drivers you should search VMware KB articles related to your setup or contact VMware for assistance. These instructions could also be used to upgrade other drivers such as HBA's, graphics accelerators, etc.

Process-2

Update Intel NIC Drivers on ESX 4.1

I Installed an IBM x3650 M3 the other day. During the installation the additional Intel NIC was not recognized by default in the ESX Host.
This I could see in two different ways, from the output on the console
msaidelk@esx9:~$ sudo lspci | grep Ethernet
0b:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
0b:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
10:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
10:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
15:00.0 Ethernet controller: Intel Corporation Unknown device 1516 (rev 01)
15:00.1 Ethernet controller: Intel Corporation Unknown device 1516 (rev 01)
1f:00.0 Ethernet controller: Intel Corporation Unknown device 1516 (rev 01)
1f:00.1 Ethernet controller: Intel Corporation Unknown device 1516 (rev 01)
And the GUI also only recognized the first 4 Broadcom NICs (instead of 8)

I posted an article about IBM x3650 M3 Does not Recognize NICs a while back - but as you can see from the output above they are recognized in hardware - just ESX does not know how to deal with them.
I downloaded the driver from VMware's Site and extracted the files from the ISO image and the file I am interested in is in the offline-bundle folder

The host has to be in Maintenance mode for the patch update.
Install through vCLI
C:\Program Files (x86)\VMware\VMware vSphere CLI\bin>vihostupdate.pl --server esx9.maishsk.local --username root  -i -b \\vc\VMware\ESX\INT-intel-lad-ddk-igb-2.4.10-offline_bundle-320657.zip
After installation
msaidelk@esx9:~$ sudo lspci | grep Ethernet0b:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
0b:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
10:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
10:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
15:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
15:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
1f:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
1f:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)Host rebooted and it comes up with all NICs recognized.

Basics Trouble shooting on VMware ESX/ESXi

1) What is management service service used for ESX/ESXi Server?
2) What is Watchdog ?
3) What is Host Agent ?

I am not able to create the VM's in  running ESX / ESXi Server, But i have some VM in ESX Server which is Running with out Any issue .

How to Solve this issue ?
[root@server]# service mgmt-vmware restart
Stopping VMware ESX Server Management services:
VMware ESX Server Host Agent Watchdog [ OK ]
VMware ESX Server Host Agent [ OK ]
Starting VMware ESX Server Management services:
VMware ESX Server Host Agent (background) [ OK ]
Availability report startup (background) [ OK ]
[root@server]# service vmware-vpxa restart
Stopping vmware-vpxa: [ OK ]
Starting vmware-vpxa: [ OK ]
[root@server]#

Basic Steps to learn Vmware ESXI Trouble shooting

When ever an issue arises on an ESX host people often rush to the Service Console and start typing various commands to figure out what is wrong. Of course some of the commands are not available with ESXi and you might not even have access to the ESXi console. Many will resort to the vCLI/vMA or even PowerCLI and that works perfectly fine. Especially the vCLI/vMA is geared towards those who have experience with ESX command-line troubleshooting. You will have all "esxcfg-*" commands to your disposal and of course resxtop; which will cover 95% of those cases where commandline details are required.

I want to stress that ESXi was not built for console access, although we do provide access to the console and it works fine. The idea around ESXi is to have a lean hypervisor which is managed from the outside versus the inside and VMware has provided multiple tools to do so. The first and foremost being of course vCenter Server or the vSphere Client. Many problems can be solved simply by using the vSphere Client connected to a host directly or through vCenter Server itself. The first KB article in the list below is a good example of how vCenter can be used to troubleshoot an inaccessible virtual machine. Something that people tend to forget is that the vSphere Client can also be used to read log files, there is no need to open up a console session for that shown below and explained in the second article in the list:

1. Open a browser and enter the URL http://, where is the IP or fully qualified domain name for the vCenter Server.
2. Provide administrative credentials when prompted.
3. Click the Browse datastores in the vCenter inventory link.
4. Navigate the webpages until you reach the appropriate datacenter, datastore, and folder as noted in step 1.
5. Click the link to the appropriate log file, and open it with your preferred editor.

On the topic of log files, for those who never worked with ESXi the location is slightly different than you are used to:

• The VMkernel, vmkwarning, and hostd logs are located at /var/log/messages
• The Host Management service (hostd = Host daemon) log is located at /var/log/vmware/hostd.log\
• The vCenter Agent log is located at /var/log/vmware/vpx/vpxa.log
• The System boot log is located at /var/log/sysboot.log
• The Automatic Availability Manager (AAM) logs are located at /var/log/vmware/aam/vmware_-xxx.log

Note that the /var/log/messages is a combination of all logs out there except for the HA log. You will need to monitor open that up seperately when troubleshooting HA related issues. Also be noted that the HA logfiles aren't part of the Syslog mechanism either unfortunately. Knowing the logfiles and the type of info you can get from it is key when troubleshooting. I encourage everyone to get familiar with it when you have the time to do so, as under pressure you don't want to find yourself fiddling around in the wrong location or logfile when you have 4 managers and your director watching over your shoulder if you have fixed it already or not.

After you have dived into the log files make sure you check the Knowledge Base. Our Knowledge Base has an excellent set of articles which can be used to troubleshoot very specific issues or at least lead you into the right direction. I have listed some of the most common issues and used KB's including a link to the article below for your convenience:

1. Restart the management agents on an ESXi host (1003490)
2. Determining why a single virtual machine is inaccessible (1018834).
3. Determining why a virtual machine was powered off or restarted (1019064).
4. Determining why multiple virtual machines are inaccessible (1019000).
5. Troubleshooting virtual machine network connection issues (1003893).
6. Interpreting virtual machine monitor and executable failures (1019471).
7. Determining why a virtual machine does not respond to user interaction at the console (1017926).
8. Using Tech Support Mode in ESXi 4.1 (1017910)
9. Determining why a VMware ESXi host is inaccessible (1019082)
10. Determining why a VMware ESXi host was powered off or restarted (1019238).
11. Determining why a VMware ESXi host does not respond to user interaction (1017135).
12. Enabling serial-line logging for an ESXi host (1003900).
13. Using performance collection tools to gather data for fault analysis (1006797).
14. Using hardware NMI facilities to troubleshoot unresponsive hosts (1014767)
15. Interpreting a VMware ESX host purple diagnostic screen (1004250).
16. Troubleshooting VMware High Availability (HA) (1001596).

In some cases however it might be required or desirable to log in to Technical Support Mode (yes this is fully supported) and work directly from the ESXi shell. The ESXi shell as many of you know also contains all esxcfg-* commands, the invaluable esxcli command and of course some shell commands that are required for troubleshooting. Some  of those commands are obvious, others are less obvious. I have listed several below to make things easier.

The one many complained about in the past but actually is available is vmkping. Vmkping can be used to do basic network troubleshooting, but also for instance to validate if jumbo frames can be used by simply adding the size of the packet:

vmkping -s 9000

One that many bumped into in the ESXi 4.0 time frame was the lack of a mount command. This mount command was actually available, but as part of busybox:

/usr/bin/busybox mount

In 4.1 though the "mount" command has been linked to busybox itself enabling you to just use "mount. The same applies to for instance fdisk. Fdisk will enable you to validate the partition setup. It has helped me many times in the past to validate that partitions were still marked as "VMFS" when someone accidentally presented VMFS volumes to Windows machines which immediately resignatured the disks. Again under 4.0 fdisk is not available as a binary but is available through "busybox", and in 4.1 is available as a link. (Most of these links are located in /usr/sbin)

/usr/bin/busybox fdisk -l

Another thing that I have done in the past regularly when I needed to evacuate a host is place the host in maintenance mode. With ESXi you can do this as follows:

vim-cmd hostsvc/maintenance_mode_enter

And of course you can also exit maintenance mode:

vim-cmd hostsvc/maintenance_mode_exit

What about listing all VMs and stopping a specific one?

vim-cmd vmsvc/getallvms

vim-cmd vmsvc/poweroff

These are just examples to show the power of vim-cmd. Many try to avoid using it, but really it is not overly complex and it gets the job done fairly simple. It can be difficult sometimes to figure out the syntax but than again if you can't find figure it someone else probably has, google it.

Something that I was asked about this week which can also come in handy when troubleshooting memory issues is the following command which will give you the memory utilization of the hypervisor components:

vdf -ph

These commands are just a couple of examples of what is possible within the ESXi shell, although we do generally recommend to avoid logging in to the ESXi shell (via remote or local tech support mode) and prefer to use the alternatives we offer it will work fine. In general troubleshooting hasn't changed much due to the full support of the "Technical Support Mode" feature, the remote command line utilities (vCLI or the vMA) and of course vCenter or the vSphere Client.

VCP5 - Upgrade a vNetwork Distributed Switch

VCP5 - Upgrade a vNetwork Distributed Switch

There are 3 versions of vNetwork Distribute Switches available: 4.0, 4.1, and 5.0. Each version provides new functionality, but also limits the interop with older versions

Upgrade Distributed Switch

1. In vSphere, browse to Networking
2. Select the switch and on the Summary tab, click Upgrade
3. Select the upgrade version and click Next
4. Confirm no hosts report as incompatible, click Next
5. Click Finish

Version Compatibility / Features

Version	Features	Compatibility
4.0	N/A	ESX 4.0 and later
4.1	Load-Based Teaming Network I/O Control	ESX 4.1 and later
5.0	User-defined network resource pools NetFlow Port Mirroring	ESX 5.0 and later

Step-1

1) How to add the ESX host to distributed switch with out downtime.

2) while upgrading the distributed switch , is there any impact on ESX hosts.

What is Fibre Channel over IP (FCIP or FC/IP)

Fibre Channel over IP (FCIP or FC/IP, also known as Fibre Channel tunneling or storage tunneling) is an Internet Protocol (IP)-based storage networking technology developed by the Internet Engineering Task Force (IETF). FCIP mechanisms enable the transmission of Fibre Channel (FC) information by tunneling data between storage area network (SAN) facilities over IP networks; this capacity facilitates data sharing over a geographically distributed enterprise. One of two main approaches to storage data transmission over IP networks, FCIP is among the key technologies expected to help bring about rapid development of the storage area network market by increasing the capabilities and performance of storage data transmission.

FCIP Versus iSCSI

The other method, iSCSI, generates SCSI codes from user requests and encapsulates the data into IP packets for transmission over an Ethernet connection. Intended to link geographically distributed SANs, FCIP can only be used in conjunction with Fibre Channel technology; in comparison, iSCSI can run over existing Ethernet networks. SAN connectivity, through methods such as FCIP and iSCSI, offers benefits over the traditional point-to-point connections of earlier data storage systems, such as higher performance, availability, and fault-tolerance. A number of vendors, including Cisco, Nortel, and Lucent have introduced FCIP-based products (such as switches and routers). A hybrid technology called Internet Fibre Channel Protocol (iFCP) is an adaptation of FCIP that is used to move Fibre Channel data over IP networks using the iSCSI protocols.

Vsphere 5 New Features

1. Storage DRS


2. Storage I/O Control for NFS

3. VMFS-5

4. ESXi Firewall

5. VMFS Scalability and Performance enhancements

6. 2TB+ pass-through RDM support

7. vCenter inventory extensibility

8. Storage APIs -- VAAI T10 Compliancy

9. Storage APIs -- VAAI Offloads for NAS

10. Storage APIs -- VAAI Thin Provisioning

11. Storage APIs -- Storage Awareness/Discovery

12. Storage APIs -- Data Protection compatible with MN

13. APD, Permanent APD Survivability Enablement

14. Snapshot enhancements

15. Storage vMotion scalability improvements

16. iSCSI Enablement: iSCSI UI Support

17. iSCSI Enablement: Stateless Support

18. Multi-queue Storage IO adapters

19. Increase NFSv3 Max Share Count to 256

20. SATA 3.0

21. Software FCoE initiator support

22. Enhanced logging support

23. Enhanced Storage metrics

24. Profile-Driven Storage

25. Storage vMotion support for snapshots

26. vSphere Storage Appliance (VSA)

27. SSD Detection and Enablement

28. vSphere Replication

29. vSphere Data Recovery 2.0

30. VADP enhancements

31. vCenter Orchestrator (vCO) Enhancements

32. vCO -- Library extension and consolidation

33. vCO -- Scalability

34. Network I/O Control (NIOC) Phase 2

35. NIOC -- User Defined Resource Pools

36. NIOC -- HBR traffic type

37. NIOC -- 802.1p tagging

38. Network Traffic Stats for iOPS

39. Improvement to UDP and Multicast traffic types

40. New networking drivers for server enablement

41. vDS support for Port mirror, LLDP and NetFlow V5

42. vDS Manage Port Group UI enhancement

43. Hot-Insert/Remove of Filters

44. Enhanced vMotion Compatibility

45. Storage vMotion support for Linked Clones

46. vMotion scalability (dual-NIC & longer latency support)

47. vNetwork API enhancements

48. vNetwork Opaque Channel

49. Support for 8 10GbE Physical NIC ports per host

50. Add Host Resources MIB to SNMP offering

51. Metro vMotion

52. Host Profile for DRS to support Stateless ESX

53. HA interop with agent VMs

54. DRS/DPM interop with agent VMs

55. DRS enhancements for Maintenance Mode

56. Enhanced processor support for FT

57. vSphere 5.0 HA aka "FDM / Fault Domain Manager"

58. vSphere HA - Heartbeat Datastores

59. vSphere HA - Support for partitions of management network

60. vSphere HA - Default isolation response changed

61. vSphere HA - New Status information in UI

62. vSphere HA - IPv6 support

63. vSphere HA - Application Awareness API publicly available

64. Extensions to create special icons for VMs

65. ESX Agent Management

66. Solution Management Plugin

67. Next-Gen vSphere Client

68. Host Profiles Enhancements

69. vCenter enhancements for stateless ESXi

70. vCenter Server Appliance

71. vCenter: Support for FileManager and VirtualDiskManager APIs

72. Virtual Hardware - Smartcard support for vSphere

73. Virtual Hardware Version 8

74. Virtual HW v8 -- 1TB VM RAM

75. Virtual HW v8 -- 32-way Virtual SMP

76. Virtual Hw v8 -- Client-Connected USB Devices

77. Virtual HW v8 -- EFI Virtual BIOS

78. Virtual HW v8 -- HD Audio

79. Virtual Hw v8 -- Multi-core Virtual CPU Support UI

80. Virtual HW v8 -- New virtual E1000 NIC

81. Virtual HW v8 -- UI and other support

82. Virtual HW v8 -- USB 3.0 device support

83. Virtual HW v8 -- VMCI device enhancements

84. Virtual HW v8 -- xHCI

85. Support SMP for Mac OS X guest OS

86. Universal Passthrough (VMdirect path with vMotion support)

87. Guest Management Operations (VIX API)

88. Guest OS Support -- Mac OS X Server

89. VM Serial Port to Host Serial Port Redirection (Serial Port Pass-Through)

90. Passthrough/SR-IOV

91. VMware Tools Portability

92. VMRC Concurrent Connections enhancements

93. Scalability: 512 VMs per host

94. ESXCLI enhancements

95. Support SAN and hw-iSCSI boot

96. Hardware -- Interlagos Processor Enablement

97. Hardware -- SandyBridge-DT Processor Enablement

98. Hardware -- SandyBridge-EN Processor Enablement

99. Hardware -- SandyBridge-EP Processor Enablement

100. Hardware -- Valencia Processor Enablement

101. Hardware -- Westmere-EX Processor Enablement

102. Platform -- CIM Enhancements

103. Platform -- ESX i18n support

104. Host Power Management Enhancements

105. Improved CPU scheduler

106. Improved scalability of CPU (NUMA) scheduler

107. Memory scheduler improvements to support 32-way VCPU's

108. Swap to host cache

109. API enhancements to configure VM boot order

110. VMX swap

111. Support for ESXi On Apple XServe

112. Redirect DCUI to host serial port for remote monitoring and management

113. UEFI BIOS Boot for ESXi hosts

114. Scalability -- 160 CPU Threads (logical PCPUs) per host

115. Scalability -- 2 TB RAM per host

116. Scalability -- 2048 VCPUs per host

117. Scalability -- 2048 virtual disks per host

118. Scalability -- 2048 VMs per VMFS volume

119. Scalability -- 512 VMs per host

120. Stateless -- Host Profile Engine and Host Profile Completeness

121. Stateless -- Image Builder

122. Stateless -- Auto Deploy

123. Stateless -- Networking Host Profile Plugin

124. Stateless -- VIB Packaging Enhancement

125. Stateless -- VMkernel network core dump

126. Host profiles enhancements for storage configuration

127. Enhanced driver support for ESXi

128. Intel TXT Support

129. Memsched policy enhancements w.r.t. Java balloon

130. Native Driver Autoload support

131. Root password entry screen in interactive installer

132. vCenter Dump Collector

133. vCenter Syslog Collector

134. VMware Update Manager (VUM) enhancements

135. VUM -- Virtual Appliance enhancements

136. VUM -- vApp Support

137. VUM -- Depot management enhancements

138. vCLI enhancements

139. PowerCLI enhancements

140. VProbes -- ESX Platform Observability

would like to add more swap space to my Linux system. Can you explain with clear examples on how to increase the swap space?

You can either use a dedicated hard drive partition to add new swap space, or create a swap file on an existing filesystem and use it as swap space.

How much swap space is currently used by the system?

Free command displays the swap space. free -k shows the output in KB.

# free -k
             total       used       free     shared    buffers     cached
Mem:       3082356    2043700    1038656          0      50976    1646268
-/+ buffers/cache:     346456    2735900
Swap:      4192956          0    4192956

Swapon command with option -s, displays the current swap space in KB.

# swapon -s
Filename                        Type            Size    Used    Priority
/dev/sda2                       partition       4192956 0       -1

Swapon -s, is same as the following.

# cat /proc/swaps
Filename                        Type            Size    Used    Priority
/dev/sda2                       partition       4192956 0       -1

Method 1: Use a Hard Drive Partition for Additional Swap Space

If you have an additional hard disk, (or space available in an existing disk), create a partition using fdisk command. Let us assume that this partition is called /dev/sdc1

Now setup this newly created partition as swap area using the mkswap command as shown below.

# mkswap /dev/sdc1

Enable the swap partition for usage using swapon command as shown below.

# swapon /dev/sdc1

To make this swap space partition available even after the reboot, add the following line to the /etc/fstab file.

# cat /etc/fstab
/dev/sdc1               swap                    swap    defaults        0 0

Verify whether the newly created swap area is available for your use.

# swapon -s
Filename                        Type            Size    Used    Priority
/dev/sda2                       partition       4192956 0       -1
/dev/sdc1                       partition       1048568 0       -2

# free -k
             total       used       free     shared    buffers     cached
Mem:       3082356    3022364      59992          0      52056    2646472
-/+ buffers/cache:     323836    2758520
Swap:      5241524          0    5241524

Note: In the output of swapon -s command, the Type column will say “partition” if the swap space is created from a disk partition.

Method 2: Use a File for Additional Swap Space

If you don’t have any additional disks, you can create a file somewhere on your filesystem, and use that file for swap space.

The following dd command example creates a swap file with the name “myswapfile” under /root directory with a size of 1024MB (1GB).

# dd if=/dev/zero of=/root/myswapfile bs=1M count=1024
1024+0 records in
1024+0 records out

# ls -l /root/myswapfile
-rw-r--r--    1 root     root     1073741824 Aug 14 23:47 /root/myswapfile

Change the permission of the swap file so that only root can access it.

# chmod 600 /root/myswapfile

Make this file as a swap file using mkswap command.

# mkswap /root/myswapfile
Setting up swapspace version 1, size = 1073737 kB

Enable the newly created swapfile.

# swapon /root/myswapfile

To make this swap file available as a swap area even after the reboot, add the following line to the /etc/fstab file.

# cat /etc/fstab
/root/myswapfile               swap                    swap    defaults        0 0

Verify whether the newly created swap area is available for your use.

# swapon -s
Filename                        Type            Size    Used    Priority
/dev/sda2                       partition       4192956 0       -1
/root/myswapfile                file            1048568 0       -2

# free -k
             total       used       free     shared    buffers     cached
Mem:       3082356    3022364      59992          0      52056    2646472
-/+ buffers/cache:     323836    2758520
Swap:      5241524          0    5241524

Note: In the output of swapon -s command, the Type column will say “file” if the swap space is created from a swap file.

If you don’t want to reboot to verify whether the system takes all the swap space mentioned in the /etc/fstab, you can do the following, which will disable and enable all the swap partition mentioned in the /etc/fstab

# swapoff -a

# swapon -a

How to Mount a Data store in ESX Server ?

You need to use the esxcfg-volume command. It can be used in this way:

• Execute this command to list the volumes that are detected as snapshots/replicas:
  
  # esxcfg-volume -l
  
  The output appears similar to:
  
  VMFS3 UUID/label: 49d22e2e-996a0dea-b555-001f2960aed8/VMFS_1
  Can mount: Yes
  Can resignature: Yes
  Extent name: naa.60a98000503349394f3450667a744245:1 range: 0 - 97023 (MB)
  
  Here the Datastore UUID is 49d22e2e-996a0dea-b555-001f2960aed8 and its last label is VMFS_1.
  
  
• To mount the volume without performing a resignaturing of that volume (this volume is unmounted when the ESX host is rebooted), run this command:
  
  # esxcfg-volume -m
  
  For example:
  
  # esxcfg-volume -m "VMFS_1"
  # esxcfg-volume -m "49d22e2e-996a0dea-b555-001f2960aed8"
  
  
• To mount the volume without performing a resignaturing of that volume (this volume is mounted when the ESX host is rebooted), run this command:
  
  # esxcfg-volume -M
  
  For example:
  
  # esxcfg-volume -M "VMFS_1"
  # esxcfg-volume -M "49d22e2e-996a0dea-b555-001f2960aed8"
  
  
• Run this command to resignature the volume (the volume is mounted immediately after the resignature):
  
  # esxcfg-volume -r
  
  For example:
  
  # esxcfg-volume -r "VMFS_1"
  # esxcfg-volume -r "49d22e2e-996a0dea-b555-001f2960aed8"

As simple questions for Vmware administrator?

1) How to find the log of ESX Server when Esx server is down?

2) How to get the  log information of HBA ?

3) I had upgraded the bios version  after that esx server is not running , how to find the log?

4)what is vilogger?

5) How to check the network information of esx server from a centralized place?

Use VMA server for all this solutions 

SOAP

SIMPLE OBJECT ACESS PROTOCOL

The VMware Host Server (SOAP) sensor monitors a VMware host server using Simple Object Access Protocol. It shows CPU (percent) and memory (absolute) usage, disk read and write speed, and network received and transmitted speed of a VMware host server

Add Sensor

The Add Sensor dialog appears when adding a new sensor on a device manually. It only shows the setting fields that are imperative for creating the sensor. Therefore, you will not see all setting fields in this dialog. You can change all settings in the sensor's Settings tab later.

VMware Host Server Sensor Settings

On the sensor's detail page, click on the Settings tab to change settings.


Note: If not set explicitly in a sensor's settings, it will connect to the IP address or DNS name defined in the settings of the parent device the sensor is created on.


Basic Sensor Settings

Sensor Name


Enter a meaningful name to identify the sensor. The name will be shown by default in the device tree and in all alarms.

Tags: Enter one or more tags, separated by space or comma. You can use tags to group sensors and use tag-filtered views later on. Tags are not case sensitive. We recommend using the default value. You can add additional tags to it, if you like. Tags are automatically inherited.

Priority

Select a priority for the sensor. This setting determines where the sensor will be placed in sensor lists. Top priority will be at the top of a list. You can choose from one star (low priority) to five stars (top priority).

Sensor Display
Primary Channel: Select a channel from the list to define it as the primary channel. In the device tree, the last value of the primary channel will always be displayed underneath the sensor's name. The available options depend on what channels are available for this sensor.


Chart Type


Define how different channels will be shown for this sensor.


•Show channels independently (default) : Show an own graph for each channel.


•Stack channels on top of each other : Stack channels on top of each other to create a multi-channel graph. This will generate an easy-to-read graph which visualizes the different components of your total traffic.


Stack Unit: This setting is only available if stacked graphs are selected above. Choose a unit from the list. All channels with this unit will be stacked on top of each other. By default, you cannot exclude single channels from stacking, if they use the selected unit. However, there is an advanced procedure to do so.


Inherited Settings:  By default, all following settings are inherited from objects higher in the hierarchy and should be changed there, if necessary. Often, best practice is to change them centrally in the Root group's settings. To change a setting for this object, disable inheritance by clicking on the check mark symbol in front of the respective setting name. You will then see the options described below.


Scanning Interval
Scanning Interval: The scanning interval determines the time the sensor waits between two scans. Select a scanning interval (seconds, minutes, or hours) from the list. You can change the available intervals in the system administration.


Schedules and Dependencies
Schedule:


Select a schedule from the list. Schedules can be used to pause monitoring for a certain time span (days, hours) throughout the week. You can create new schedules and edit existing ones in the account settings. Note: Schedules are generally inherited. New schedules will be added to existing ones, so all schedules are active.


Inherit Access Rights
User Group Access
Define which user group(s) will have access to the object you're editing. A table with user groups and right is shown; it contains all user groups from your setup. For each user group you can choose from the following access rights:


•Inherited : Use the settings of the parent object.


•None : Users in this group cannot see or edit the object. The object does not show up in lists and in the sensor tree. Exception: If a child object is visible to the user, the object is visible in the sensor tree, though not accessible.


•Read : Users in this group can see the object and review its monitoring results.


•Write : Users in this group can see the object, review its monitoring results, and edit the object's settings. They cannot edit access rights settings.


•Full : Users in this group can see the object, review its monitoring results, edit the object's settings, and edit access rights settings.


You can create new user groups in the System Administration—User Groups settings. To automatically set all objects further down in the hierarchy to inherit this object's access rights, set a check mark for the Revert children's access rights to inherited option