ZFS Cheatsheet
Updated on 21 Aug '08 - Added section on sharing information following a comment.
Updated on 11 Sept '07 - Updated to show functionality available in Nevada build 71.
This came round on one of the many internal aliases - Thanks Jim Laurent
The three primary goals of ZFS are:
- Highly scalable (128-bit) data repository
- Ease of administration
- Guaranteed on disk data integrity
Source code and community support is available at https://www.opensolaris.org/os/community/zfs/
Sample ZFS commands and usage
| What You Do and See | Why |
|---|---|
$ man zpool
$ man zfs
|
Get familiar with command structure and options |
$ su
Password:
# cd /
# mkfile 100m disk1 disk2 disk3 disk5
# mkfile 50m disk4
# ls -l disk*
-rw------T 1 root root 104857600 Sep 11 12:15 disk1
-rw------T 1 root root 104857600 Sep 11 12:15 disk2
-rw------T 1 root root 104857600 Sep 11 12:15 disk3
-rw------T 1 root root 52428800 Sep 11 12:15 disk4
-rw------T 1 root root 104857600 Sep 11 12:15 disk5
|
Create some "virtual devices" or vdevs as described in the zpool documentation. These can also be real disk slices if you have them available. |
# zpool create myzfs /disk1 /disk2
# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
myzfs 191M 94K 191M 0% ONLINE -
|
Create a storage pool and check the size and usage. |
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
errors: No known data errors
|
Get more detailed status of the zfs storage pool. |
# zpool destroy myzfs
# zpool list
no pools available
|
Destroy a zfs storage pool |
# zpool create myzfs mirror /disk1 /disk4
invalid vdev specification
use '-f' to override the following errors:
mirror contains devices of different sizes
|
Attempt to create a zfs pool with different size vdevs fails. Using -f options forces it to occur but only uses space allowed by smallest device. |
# zpool create myzfs mirror /disk1 /disk2 /disk3
# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
myzfs 95.5M 112K 95.4M 0% ONLINE -
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
/disk3 ONLINE 0 0 0
errors: No known data errors
|
Create a mirrored storage pool. In this case, a 3 way mirrored storage pool. |
# zpool detach myzfs /disk3
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
errors: No known data errors
|
Detach a device from a mirrored pool. |
# zpool attach myzfs /disk1 /disk3
# zpool status -v
pool: myzfs
state: ONLINE
scrub: resilver completed with 0 errors on Tue Sep 11 13:31:49 2007
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
/disk3 ONLINE 0 0 0
errors: No known data errors
|
Attach device to pool. This creates a two-way mirror is the pool is not already a mirror, else it adds another mirror, in this case making it a 3 way mirror. |
# zpool remove myzfs /disk3
cannot remove /disk3: only inactive hot spares can be removed
# zpool detach myzfs /disk3
|
Attempt to remove a device from a pool. In this case it's a mirror, so we must use "zpool detach". |
# zpool add myzfs spare /disk3
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
spares
/disk3 AVAIL
errors: No known data errors
|
Add a hot spare to a storage pool. |
# zpool remove myzfs /disk3
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
errors: No known data errors
|
Remove a hot spare from a pool. |
# zpool offline myzfs /disk1
# zpool status -v
pool: myzfs
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning
in a degraded state.
action: Online the device using 'zpool online' or replace the device
with 'zpool replace'.
scrub: resilver completed with 0 errors on Tue Sep 11 13:39:25 2007
config:
NAME STATE READ WRITE CKSUM
myzfs DEGRADED 0 0 0
mirror DEGRADED 0 0 0
/disk1 OFFLINE 0 0 0
/disk2 ONLINE 0 0 0
errors: No known data errors
|
Take the specified device offline. No attempt to read or write to the device will take place until it's brought back online. Use the -t option to temporarily offline a device. A reboot will bring the device back online. |
# zpool online myzfs /disk1
# zpool status -v
pool: myzfs
state: ONLINE
scrub: resilver completed with 0 errors on Tue Sep 11 13:47:14 2007
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
errors: No known data errors
|
Bring the specified device online. |
# zpool replace myzfs /disk1 /disk3
# zpool status -v
pool: myzfs
state: ONLINE
scrub: resilver completed with 0 errors on Tue Sep 11 13:25:48 2007
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk3 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
errors: No known data errors
|
Replace a disk in a pool with another disk, for example when a disk fails |
# zpool scrub myzfs
|
Perform a scrub of the storage pool to verify that it checksums correctly. On mirror or raidz pools, ZFS will automatically repair any damage. WARNING: scrubbing is I/O intensive. |
# zpool export myzfs
# zpool list
no pools available
|
Export a pool from the system for importing on another system. |
# zpool import -d / myzfs
# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
myzfs 95.5M 114K 95.4M 0% ONLINE -
|
Import a previously exported storage pool. If -d is not specified, this command searches /dev/dsk. As we're using files in this example, we need to specify the directory of the files used by the storage pool. |
# zpool upgrade
This system is currently running ZFS pool version 8.
All pools are formatted using this version.
# zpool upgrade -v
This system is currently running ZFS pool version 8.
The following versions are supported:
VER DESCRIPTION
--- --------------------------------------------------------
1 Initial ZFS version
2 Ditto blocks (replicated metadata)
3 Hot spares and double parity RAID-Z
4 zpool history
5 Compression using the gzip algorithm
6 pool properties
7 Separate intent log devices
8 Delegated administration
For more information on a particular version, including supported
releases, see:
https://www.opensolaris.org/os/community/zfs/version/N
Where 'N' is the version number.
|
Display pools format version. The -v flag shows the features supported by the current version. Use the -a flag to upgrade all pools to the latest on-disk version. Pools that are upgraded will no longer be accessible to any systems running older versions. |
# zpool iostat 5
capacity operations bandwidth
pool used avail read write read write
---------- ----- ----- ----- ----- ----- -----
myzfs 112K 95.4M 0 4 26 11.4K
myzfs 112K 95.4M 0 0 0 0
myzfs 112K 95.4M 0 0 0 0
|
Get I/O statistics for the pool |
# zfs create myzfs/colin
# df -h
Filesystem kbytes used avail capacity Mounted on
...
myzfs/colin 64M 18K 63M 1% /myzfs/colin
|
Create a file system and check it with standard df -h command. File systems are automatically mounted by default under the /zfs location. See the Mountpoints section of the zfs man page for more details. |
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 139K 63.4M 19K /myzfs
myzfs/colin 18K 63.4M 18K /myzfs/colin
|
List current zfs file systems. |
# zpool add myzfs /disk1
invalid vdev specification
use '-f' to override the following errors:
mismatched replication level: pool uses mirror and new vdev is file
|
Attempt to add a single vdev to a mirrored set fails |
# zpool add myzfs mirror /disk1 /disk5
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
myzfs ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk3 ONLINE 0 0 0
/disk2 ONLINE 0 0 0
mirror ONLINE 0 0 0
/disk1 ONLINE 0 0 0
/disk5 ONLINE 0 0 0
errors: No known data errors
|
Add a mirrored set of vdevs |
# zfs create myzfs/colin2
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 172K 159M 21K /myzfs
myzfs/colin 18K 159M 18K /myzfs/colin
myzfs/colin2 18K 159M 18K /myzfs/colin2
|
Create a second file system. Note that both file system show 159M available because no quotas are set. Each "could" grow to fill the pool. |
# zfs set reservation=20m myzfs/colin
# zfs list -o reservation
RESERV
none
20M
none
|
Reserve a specified amount of space for a file system ensuring that other users don't take up all the space. |
# zfs set quota=20m myzfs/colin2
# zfs list -o quota myzfs/colin myzfs/colin2
QUOTA
none
20M
|
Set and view quotas |
# zfs set compression=on myzfs/colin2
# zfs list -o compression
COMPRESS
off
off
on
|
Turn on and verify compression |
# zfs set sharenfs=on myzfs/colin2
# zfs get sharenfs myzfs/colin2
NAME PROPERTY VALUE SOURCE
myzfs/colin2 sharenfs on local
|
Share a filesystem over NFS. There is no need to modify the /etc/dfs/dfstab as the filesystem will be share automatically on boot. |
# zfs set sharesmb=on myzfs/colin2
# zfs get sharesmb myzfs/colin2
NAME PROPERTY VALUE SOURCE
myzfs/colin2 sharesmb on local
|
Share a filesystem over CIFS/SMB. This will make your ZFS filesystem accessible to Windows users. |
# zfs snapshot myzfs/colin@test
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 20.2M 139M 21K /myzfs
myzfs/colin 18K 159M 18K /myzfs/colin
myzfs/colin@test 0 - 18K -
myzfs/colin2 18K 20.0M 18K /myzfs/colin2
|
Create a snapshot called test. |
# zfs rollback myzfs/colin@test
|
Rollback to a snapshot. |
# zfs clone myzfs/colin@test myzfs/colin3
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 20.2M 139M 21K /myzfs
myzfs/colin 18K 159M 18K /myzfs/colin
myzfs/colin@test 0 - 18K -
myzfs/colin2 18K 20.0M 18K /myzfs/colin2
myzfs/colin3 0 139M 18K /myzfs/colin3
|
A snapshot is not directly addressable. A clone must be made. The target dataset can be located anywhere in the ZFS hierarchy, and will be created as the same type as the original. |
# zfs destroy myzfs/colin2
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 20.1M 139M 22K /myzfs
myzfs/colin 18K 159M 18K /myzfs/colin
myzfs/colin@test 0 - 18K -
myzfs/colin3 0 139M 18K /myzfs/colin3
|
Destroy a filesystem |
# zfs destroy myzfs/colin
cannot destroy 'myzfs/colin': filesystem has children
use '-r' to destroy the following datasets:
myzfs/colin@test
|
Attempt to destroy a filesystem that had a child. In this case, the snapshot filesystem. We must either remove the snapshot, or make a clone and promote the clone. |
# zfs promote myzfs/colin3
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 20.1M 139M 21K /myzfs
myzfs/colin 0 159M 18K /myzfs/colin
myzfs/colin3 18K 139M 18K /myzfs/colin3
myzfs/colin3@test 0 - 18K -
# zfs destroy myzfs/colin
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 147K 159M 21K /myzfs
myzfs/colin3 18K 159M 18K /myzfs/colin3
myzfs/colin3@test 0 - 18K -
|
Promte a clone filesystem to no longer be a dependent on it's "origin" snapshot. This now associates makes the snapshot a child of the cloned filesystem. We can then delete the original filesystem. |
# zfs rename myzfs/colin3 myzfs/bob
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 153K 159M 21K /myzfs
myzfs/bob 18K 159M 18K /myzfs/bob
myzfs/bob@test 0 - 18K -
# zfs rename myzfs/bob@test myzfs/bob@newtest
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 146K 159M 20K /myzfs
myzfs/bob 18K 159M 18K /myzfs/bob
myzfs/bob@newtest 0 - 18K -
|
Rename a filesystem, and separately rename the snapshot. |
# zfs get all
NAME PROPERTY VALUE SOURCE
myzfs type filesystem -
myzfs creation Tue Sep 11 14:21 2007 -
myzfs used 146K -
myzfs available 159M -
myzfs referenced 20K -
[...]
|
Display properties for the given datasets. This can be refined further using options. |
# zpool destroy myzfs
cannot destroy 'myzfs': pool is not empty
use '-f' to force destruction anyway
|
Can't destroy a pool with active filesystems. |
# zfs unmount myzfs/bob
# df -h
myzfs 159M 20K 159M 1% /myzfs
|
Unmount a ZFS file system |
# zfs mount myzfs/bob
# df -h
myzfs 159M 20K 159M 1% /myzfs
myzfs/bob 159M 18K 159M 1% /myzfs/bob
|
Mount a ZFS filesystem. This is usually automatically done on boot. |
# zfs send myzfs/bob@newtest | ssh localhost zfs receive myzfs/backup
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
myzfs 172K 159M 20K /myzfs
myzfs/backup 18K 159M 18K /myzfs/backup
myzfs/backup@newtest 0 - 18K -
myzfs/bob 18K 159M 18K /myzfs/bob
myzfs/bob@newtest 0 - 18K -
|
Create a stream representation of the snapshot and redirect it to zfs receive. In this example I've redirected to the localhost for illustration purposes. This can be used to backup to a remote host, or even to a local file. |
# zpool history
History for 'myzfs':
2007-09-11.15:35:50 zpool create myzfs mirror /disk1 /disk2 /disk3
2007-09-11.15:36:00 zpool detach myzfs /disk3
2007-09-11.15:36:10 zpool attach myzfs /disk1 /disk3
2007-09-11.15:36:53 zpool detach myzfs /disk3
2007-09-11.15:36:59 zpool add myzfs spare /disk3
2007-09-11.15:37:09 zpool remove myzfs /disk3
2007-09-11.15:37:18 zpool offline myzfs /disk1
2007-09-11.15:37:27 zpool online myzfs /disk1
2007-09-11.15:37:37 zpool replace myzfs /disk1 /disk3
2007-09-11.15:37:47 zpool scrub myzfs
2007-09-11.15:37:57 zpool export myzfs
2007-09-11.15:38:05 zpool import -d / myzfs
2007-09-11.15:38:52 zfs create myzfs/colin
2007-09-11.15:39:27 zpool add myzfs mirror /disk1 /disk5
2007-09-11.15:39:38 zfs create myzfs/colin2
2007-09-11.15:39:50 zfs set reservation=20m myzfs/colin
2007-09-11.15:40:18 zfs set quota=20m myzfs/colin2
2007-09-11.15:40:35 zfs set compression=on myzfs/colin2
2007-09-11.15:40:48 zfs snapshot myzfs/colin@test
2007-09-11.15:40:59 zfs rollback myzfs/colin@test
2007-09-11.15:41:11 zfs clone myzfs/colin@test myzfs/colin3
2007-09-11.15:41:25 zfs destroy myzfs/colin2
2007-09-11.15:42:12 zfs promote myzfs/colin3
2007-09-11.15:42:26 zfs rename myzfs/colin3 myzfs/bob
2007-09-11.15:42:57 zfs destroy myzfs/colin
2007-09-11.15:43:23 zfs rename myzfs/bob@test myzfs/bob@newtest
2007-09-11.15:44:30 zfs receive myzfs/backup
|
Display the command history of all storage pools. This can be limited to a single pool by specifying its name on the command line. The history is only stored for existing pools. Once you've destroyed the pool, you'll no longer have access to it's history. |
# zpool destroy -f myzfs
# zpool status -v
no pools available
|
Use the -f option to destroy a pool with files systems created. |