I was working on masking some storage paths some days ago and after finding the VMware KB I realized that I needed to create a blog post about storage and how it relates to VMware before I go and explain how to mask paths and why its is done.
This will be the first of three posts regarding storage in VMware, leading up to a explanation of masking storage paths in VMware.
You can put the components in a VMware storage array connection in small list:
- LUN ID.
- Datastore UUID.
- LUN Paths.
- Adapter identifier
- Target identifier (Storage array)
1. Every LUN has an ID to identify between them.
- NAA & EUI: Most common are the NAA, that’s Network Address Authority Identifier, or a EUI, that’s Extended Unique Identifier.
- MPX : Local Datastores have another UUID, for example mpx.vmhba33:C0:T1:L0. This is only for local LUNs and can be used in the exact same way as the NAA identifier.
- VML: Can be used interchangeably with the NAA identifier and the MPX identifier. Used for operations such as vmkfstools.
2 .Every VMFS volume has UUID, a Universally Unique Identifier that is assigned at creation.
3 .Then you got Paths to a LUN. Thats the connections from the ESXi host to the Storage Array.
Most commonly you have 4-8 paths, at least 2 to have some kind of failover.
Paths are identified with something like this: vmhba1:C0:T1:L1. What does these numbers mean?
- vmhba1= Name of the storage adapter on the physical ESXi host. Most commonly you got two, vmhba1 and vmhba2.
- C=Channel, or the storage channel. Some controllers support multiple connections to the same controller.
- T=Target, this is a number made up by the host for the Datastore’s paths, so in a 4 path Datastore (with 2 HBAs) you would get :vmhba1:C0:T0:L1,vmhba1:C0:T1:L1,vmhba2:C0:T0:L1,vmhba2:C0:T1:L1.
- Please note that these target number are NOT shared between hosts and can change between reboots. Really its the NAA(or EUI) that lets you know what datastore you are referring to.
- L=LUN number. A number provided by the storage system.
4. Adapter identifier is made from the WWNN (world wide node name) and WWPN (world wide port name). The format is fc.200100e08ba5ff63:210100e08ba5ff63. you can see this in the vSphere client:
This is a ID (WWN) for the ESXi host adapter. This adapter ID is used mainly when zoning storage arrays and hosts together.
I want to explain what you see in the picture:
First you have the Paths (vmhba2:C….), as you can see this datastore has 4 paths to it (8 if the dead ones are counted…).
All paths has a different Target identifier as this Storage array has 4 HBA cards (IBM SVC).
5. Target identifier has the same format as the adapter identifier. Can also be seen in the vSphere client.
This is an ID for the storage array. Also used mainly in zoning.
Finally I wanted to add a video how these storage components work together.
*Sidenote : Most common ESXi 5.0 CLI commands to get some storage data are:
- esxcli storage core path list : Generates a list of all LUN paths currently connected to the ESX host.
- esxcli storage core device list: Generates a list of LUN’s currently connected to the ESX host.
- esxcli storage vmfs extent list : Generates a list of extents for each volume as well as providing the mapping from the device name to the UUID.
- esxcli storage filesystem list: Generates a compact list of the LUNs currently connected to the ESX host, including VMFS version.
- ls -alh /vmfs/devices/disks: Lists the possible targets for certain storage operations.
Next post will go deeper into VMware PSA (Pluggable Storage Arcitecture) and how it works.