2006-11-13 · in Ideas · 273 words

One of the great -- or irritating -- things about using free Unix-like systems is the wide variety of filesystems available. At the moment, though, it's hard to experiment with new filesystems because there's no good way of converting one format to another in place. I'm proposing a generic way of doing this.

You'd need to start with the "source" filesystem mounted read-only, so that it was guaranteed not to change during conversion. The format converter would provide a virtual writable block device upon which the "target" filesystem would be created and mounted; any normal tool could be used to copy the files from the source filesystem to the target filesystem.

The trick is that the vast majority of the data is going to be the same on the old and new filesystems; it's just the location on disk that changes. We can therefore use a similar trick to the rsync protocol: as the copy operation writes the new files to the target filesystem (mounted on the virtual block device), the converter makes a log of where the data that's being written was actually copied from by searching the source device (perhaps disabling read caching and using a log of recent accesses to speed up the search). Once the copy's finished, we know which data was copied from areas of the source disk and which data was constructed from scratch; it's then a reasonably easy operation to reshuffle data on the source disk to match the target layout.

The same approach can be used for defragmentation (or general layout optimisation); you'd just create the same kind of filesystem on the target as on the source.