Identifying and removing inaccessible vSAN objects

Once in a while and for various reasons, a vSAN object will become inaccessible. Problem is: The inaccessible objects occupies space on the vSAN Datastore, but is not counted in overall capacity use as seen by Skyline health and Capacity. I recently dealt with just such an issue where Skyline Health and Capacity were showing 32% available vSAN capacity, but in Datastores the actually availability was just a few hundred GB, less than 5% of the vSAN capacity.

As it turns out, this was because of a large inaccessible object (also determined to be not needed/obsolete) which was visible in vSAN Virtual Objects.

Here’s the solution:

  1. Identify the UID of the inaccessible object in vCenter, under Virtual Objects
  2. Log in to any ESXi Host in the cluster with SSH
    1. Run the following command to identify the friendly name and properties of the inaccessible object:
      1. /usr/lib/vmware/osfs/bin/objtool getAttr -u <OBJECT UID>
    2. If that fails to retrieve any info, run this command against the UID of the inaccessible object:
      1. /usr/lib/vmware/osfs/bin/objtool getAttr -u <OBJECT UID> –bypassDom
  3. Verify the friendly name that the inaccessible object belonged to and, if appropriate, proceed to remove that object
    1. /usr/lib/vmware/osfs/bin/objtool delete -u <OBJECT UID> -f

Here’s what it looks like in practice:

[root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool getAttr -u f47e275f-8cc3-f865-c458-00a38e8e2fea
Failed to get object attributes : Input/output error 327684.
object getAttr error: Failure
[root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool getAttr -u f47e275f-8cc3-f865-c458-00a38e8e2fea --bypassDom
Object Attributes --


Object type:vsan

Object size:32212254720000

User friendly name:(null)

HA metadata:(null)

Allocation type:Thin

Policy:((\"stripeWidth\" i1) (\"cacheReservation\" i0) (\"proportionalCapacity\" i0) (\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i0) (\"spbmProfileId\" \"1d04bc75-ffe7-4316-adbf-ddc8f2168bb7\") (\"spbmProfileGenerationNumber\" l+0) (\"replicaPreference\" \"Capacity\") (\"iopsLimit\" i0) (\"checksumDisabled\" i0) (\"spbmProfileName\" \"Backup Volume\"))

Object class: vdisk

Object capabilities: NONE

Object path: /vmfs/volumes/vsan:526186362509cc06-d35c848bf26e6e22/71363c5e-6c76-e8a7-df90-00be75494ac8/old_veeam_repo_1_1.vmdk

Group uuid: 71363c5e-6c76-e8a7-df90-00be75494ac8

Container uuid: (null)

[root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool delete -u f47e275f-8cc3-f865-c458-00a38e8e2fea -f

About: John Borhek

John Borhek is the CEO and Lead Solutions Architect at VMsources Group Inc. John has soup-to-nuts experience in Mission Critical Infrastructure, specializing in hyper-convergence and Cloud Computing, engaging with organizations all over the United States and throughout the Americas.

Leave a Reply

Your email address will not be published. Required fields are marked *