Hi everyone
I have an ESX 6.7 which consists of a datastore named Vira nad this datastore was built over a 1TB Samsung SSD. after a while we needed to expand it and did it using another 1TB and expanded its storage space to about 2TB. last week we had a bad hardware restart and after restart Datastore named Vira had some issues. the first problem was its naming and I figured out that Datastore is not available correctly. I tried to power on a virtual machine that corresponds to this datastore and it didn't come up. after some investigation i found one of the disks included in 2TB Extent is not available. tried to change the missing disk SATA port and it was available in system again. after starting the ESX I could see the Datastore but after that change and start I cannot power on the machin and FLAT VMDK file is locked. every time I try to clone it, touch it , move it I get Invalid Argument error.
searche a lot in VMWare KBs and Forums and found it may be some process and checked it using VMKFSTOOLS -D. it has a lock that relates to ESX itself. restarting Services or even Host doesn't remove the lock.
vmkfstools -D W-Sharing\ SRV_2-flat.vmdk
Lock [type 10c00001 offset 134742016 v 8, hb offset 3702784
gen 15, mode 1, owner 57a3177f-dbdbe28c-4845-002590cb29c2 mtime 82378
num 0 gblnum 0 gblgen 0 gblbrk 0]
Addr <4, 52, 0>, gen 1, links 1, type reg, flags 0x9, uid 0, gid 0, mode 600
len 1869169766400, nb 1299233 tbz 0, cow 0, newSinceEpoch 1299233, zla 3, bs 1048576
affinityFD <4,52,0>, parentFD <4,35,0>, tbzGranularityShift 20, numLFB 0
lastSFBClusterNum 2364, numPreAllocBlocks 0, numPointerBlocks 218
tried to touch the file :
touch: W-Sharing SRV_2-flat.vmdk: Invalid argument
tried unmounting and checking the Datastore using VOMA and found these results:
here is the extent list when Datastore Vira is mounted:
esxcli storage vmfs extent list
Volume Name VMFS UUID Extent Number Device Name Partition
----------- ----------------------------------- ------------- ------------------------------------------------------------------------ ---------
All OS 57892a55-88ac4e8a-d55d-002590cb29c2 0 t10.ATA_____Samsung_SSD_860_EVO_1TB_________________S4BDNG0M104285X_____ 3
Kiantech 57a5fb65-e2d18e7a-36f6-002590cb29c2 0 t10.ATA_____Samsung_SSD_840_EVO_1TB_________________S1D9NEADA02146Y_____ 1
Wiera1 5849d59d-f80d0f04-5805-002590cb29c2 0 t10.ATA_____Samsung_SSD_860_QVO_2TB_________________S4CYNF0M207684P_____ 1
Vira 57a5fac4-a0e012b6-91f4-002590cb29c2 0 t10.ATA_____Samsung_SSD_840_EVO_1TB_________________S1D9NEADA02133Y_____ 1
Vira 57a5fac4-a0e012b6-91f4-002590cb29c2 0 t10.ATA_____Samsung_SSD_840_EVO_1TB_________________S1D9NEADB01872B_____ 1
and here is the result when Vira is not mounted:
esxcli storage vmfs extent list
Volume Name VMFS UUID Extent Number Device Name Partition
----------- ----------------------------------- ------------- ------------------------------------------------------------------------ ---------
All OS 57892a55-88ac4e8a-d55d-002590cb29c2 0 t10.ATA_____Samsung_SSD_860_EVO_1TB_________________S4BDNG0M104285X_____ 3
Kiantech 57a5fb65-e2d18e7a-36f6-002590cb29c2 0 t10.ATA_____Samsung_SSD_840_EVO_1TB_________________S1D9NEADA02146Y_____ 1
Wiera1 5849d59d-f80d0f04-5805-002590cb29c2 0 t10.ATA_____Samsung_SSD_860_QVO_2TB_________________S4CYNF0M207684P_____ 1
Vira 57a5fac4-a0e012b6-91f4-002590cb29c2 0 t10.ATA_____Samsung_SSD_840_EVO_1TB_________________S1D9NEADA02133Y_____ 1
and here is the VOMA result itself:
voma -m vmfs -f check -d /vmfs/devices/disks/t10.ATA_____Samsung_SSD_840_EVO_1TB_________________S1D9NEADA02133Y_____:1
Running VMFS Checker version 2.1 in check mode
Initializing LVM metadata, Basic Checks will be done
Checking for filesystem activity
Performing filesystem liveness check..|Scanning for VMFS-6 host activity (4096 bytes/HB, 1024 HBs).
Phase 1: Checking VMFS header and resource files
Detected VMFS-6 file system (labeled:'Vira') with UUID:57a5fac4-a0e012b6-91f4-002590cb29c2, Version 6:82
ERROR: Trying to do IO beyond device Size
ERROR: Failed to check fbb.sf.
VOMA failed to check device : Limit exceeded
Total Errors Found: 0
Kindly Consult VMware Support for further assistance
can you please help me investigate the problem over this deployment ?
thanks everyone
Mohammad