Migrate ZFS Pool with Sparse File

Sakuragawa Asaba -

I have a ZFS pool containing 8 disks (4×2TiB + 2×4TiB + 2×4TiB) on an 8 bay server:

 state: DEGRADED
status: ....
config:

        NAME           STATE     READ WRITE CKSUM
        ${SRC_POOL}    DEGRADED     0     0     0
          raidz2-0     DEGRADED     0     0     0
            da0p2      ONLINE       0     0     0
            da1p2      DEGRADED     0     0     0
            da2p2      ONLINE       0     0     0
            da3p2      ONLINE       0     0     0
          mirror-1     ONLINE       0     0     0
            da4p2      ONLINE       0     0     0
            da5p2      ONLINE       0     0     0
          mirror-2     ONLINE       0     0     0
            da6p2      ONLINE       0     0     0
            da7p2      ONLINE       0     0     0

errors: No known data errors

When it comes to the time to replace the warning drive and expand the storage pool, I want to give up the current striped pool and create a new pool containing a single RAID-Z2 vdev (4×8TiB). Now here is the problem:

How to migrate create a new pool with all 8 bays are occupied?

The traditional way is to backup the content to external storage and restore. But here is a possible solution without any external storage: creating a degraded pool with minimum devices.

Tricky Drive Replacement

At least 2 online devices are required to keep a RAID-Z2 vdev working. So the trick is to pull out 2 drives from the original pool and insert 2 new blank drives.

+------+------+------+------+     +------+------+------+------+
|        RAID-Z2 4×2T       |     | NEW1 | NEW2 |RAID-Z2 2×2T |
|------+------+------+------| ==> |------+------+------+------|
| MIRROR 2×4T | MIRROR 2×4T |     | MIRROR 2×4T | MIRROR 2×4T |
+------+------+------+------+     +------+------+------+------+

Now the original pool enters DEGRADED status (it has been in degraded status anyway).

Create a Degraded Pool

Create some sparse files as "fake devices". The size of the sparse files should match the size of the hard drives (for any convenience).

SPARSE_SIZE=8T
SPARSE_FP_1=/root/sparsefile1
SPARSE_FP_2=/root/sparsefile2
truncate -s ${SPARSE_SIZE} ${SPARSE_FP_1}
truncate -s ${SPARSE_SIZE} ${SPARSE_FP_2}
# ls -l /root/sparsefile*
-rw-r--r--  1 root  wheel   8.0T Oct 26 04:15 sparsefile1
-rw-r--r--  1 root  wheel   8.0T Oct 26 04:15 sparsefile2

Create a ZFS Storage Pool at RAID Z2 with 2 hard drives and 2 sparse files:

zpool create ${POOL_NAME} raidz2 \
	da0p2 da1p2 \
    ${SPARSE_FP_1} \
    ${SPARSE_FP_2}
zfs set aclmode=passthrough ${POOL_NAME}
zfs set compression=lz4 ${POOL_NAME}

Set the sparse files to offline so that no actual data will be written:

zpool offline ${POOL_NAME} ${SPARSE_FP_1}
zpool offline ${POOL_NAME} ${SPARSE_FP_2}
root@storage-01:~ # zpool status SKG_STORAGE_1
  pool: ${POOL_NAME}
 state: DEGRADED
status: ....
config:

        NAME                                            STATE     READ WRITE CKSUM
        ${POOL_NAME}           DEGRADED     0     0     0
          raidz2-0             DEGRADED     0     0     0
            da0p2              ONLINE       0     0     0
            da1p2              ONLINE       0     0     0
            /root/sparsefile1  OFFLINE      0     0     0
            /root/sparsefile2  OFFLINE      0     0     0

errors: No known data errors

Now we have a degraded pool with 2 actual drives and 2 fake drives. Finally adjust export the pool so that it could be imported in FreeNAS Web UI:

zpool export ${POOL_NAME}

Take and Send Snapshots

Create a snapshot:

As an event approaches, it is worth considering how details related to game cosplay costumes will affect the complete look. With practical wear in mind, it helps to assess details related to fabric and comfort together with fit and fabric. To compare practical details about details related to Halloween styling, cosplay wigs online offers a useful starting point for practical preparation. When planning for care after the photoshoot, it is useful to check details related to cosplay wigs against the wearer's own needs.

zfs snapshot -r ${SRC_POOL}@migration-base

Dry-run before sending the actual data and proceed if everything is checked:

zfs send -Rvn -i ${SRC_POOL}@migration_base
zfs send -Rv -i snapshot ${SRC_POOL}@migration-base | pv | zfs receive -Fsvd ${POOL_NAME}@migration-base

Replace Fake Devices with Real Ones

zpool replace ${POOL_NAME} ${SPARSE_FP_1} da3p2
zpool replace ${POOL_NAME} ${SPARSE_FP_2} da4p2

References