GPFS on AIX–Basics & PV Migration

Basics

Filesets

gpfs.base - GPFS File Manager
gpfs.msg.en_US - GPFS Server Messages - U.S. English
gpfs.docs.data - GPFS Server Manages and Documentation

Path

GPFS commands are in a separate directory.

export PATH=$PATH:/usr/lpp/mmfs/bin

Daemons, status

# ps -ef | grep mmfs
# mmlscluster
# mmlsconfig

Log

# tail -f /var/adm/ras/mmfs.log.latest

PV migration

Collect information

Dump/Save current configuration 

# gpfs.snap

List active mounts

# mmlsmount all

File system foobar is mounted on 2 nodes. 

List storage pools

# mmlsfs all -P

File system attributes for /dev/foobar: 

=====================================
flag value            description
---- ---------------- -----------------------------------------------------
-P  system           Disk storage pools in file system

List disks in each filesystem

# mmlsfs all -d
File system attributes for /dev/foobar:  =====================================
flag value            description
---- ---------------- -----------------------------------------------------
-d  mycluster00nsd    Disks in file system

List current NSDs (network shared disks)

# mmlsnsd -M
Disk name    NSD volume ID      Device         Node name                Remarks
---------------------------------------------------------------------------------------
mycluster00nsd 0AEC13994BFCEEF7   /dev/hdisk7    host1.mydomain.com
mycluster00nsd 0AEC13994BFCEEF7   /dev/hdisk7    host1.mydomain.com
mycluster00nsd 0AEC13994BFCEEF7   -              host3.mydomain.com    (not found) directly attached
mmlsnsd: 6027-1370 The following nodes could not be reached:
host3.mydomain.com

List filesystem manager node(s)

# mmlsmgr
file system      manager node
---------------- ------------------
foobar             10.111.11.111 (host2)
Cluster manager node: 10.111.11.111 (host2)

-Show the state of GPFS daemons on the local node:

# mmgetstate
Node number  Node name        GPFS state
------------------------------------------
       2      myhost2          active

- on all  cluster members:

# mmgetstate -a
Node number  Node name        GPFS state
------------------------------------------
       1      myhost1          active
       2      myhost2          active
       3      somexx1          unknown

Add new disk(s)

Configure new device 

# lspv
# cfgmgr
Verify new disk
# lspv
# lspath -l hdiskX
# errpt | head

Edit desc file

Disk name: hdiskX
Primary/Backup server: only for NFS
Disk usage: almost always 'dataAndMetadata'
Failure group: can be -1 when there are no other disks for failover, otherwise see 'mmdf $fs' -> failure group
Desired name: arbitrary; 'clusternameXXnsd' is suggested
Storage pool(s): see 'mmlsfs all -P'

The contents of the file is similar to this: 

#DiskName:PrimaryServer:BackupServer:DiskUsage:FailureGroup:DesiredName:StoragePool
hdisk7:::dataAndMetadata:-1:foobar00nsd:system

Configure new disk(s) as NSD

# mmcrnsd -F /path/to/hdisk7.desc

Check NSDs

# mmlsnsd; mmlsnsd -F

Add disk to FS using the transformed desc file (mmcrnsd comments out the previous line and inserts a new one)

# mmadddisk foobar -F hdisk7.desc