|
ZFS: Should You? With Sun’s new file system, ZFS, the world will never be the same again. Well the computing world, at least. There's many quick "howto" articles on ZFS already. We’d like to expand on ZFS now and talk about some limitations, performance considerations, and manageability aspects that should aid in the decision making process, in case you were thinking of switching.
Sites with large amounts of storage, say .5 PB or more, use Veritas almost without exception. Their volume manager, VxVM, combined with the file system, VxFS, makes storage management very robust and easy to handle, compared with native OS file systems and volume managers. That is, until ZFS. The remaining advantage for VxVM/FS is that you can use the same volumes on multiple platforms. AIX, Linux, Solaris and others can all use the same file system, and frequently the Veritas file system outperforms native alternatives. Having the ability to use the same file system across multiple platforms is important, especially when a SAN is involved. Migrating services to new servers on different platforms is far simpler when there’s no need to worry about file systems. ZFS outperforms Veritas VxVM/FS for most uses, but ZFS is only available on Solaris, for the time being. However, ZFS is far more flexible and easier to manage—more on that in a bit. Performance Performance is an important consideration for file system selection. Running a general file server (perhaps for home directories), a mail server, and an Oracle database are all very different problems that demand certain optimizations at the file system level. Advanced file systems layered on top of a volume manager allow the setting of the block size, or Record Size as ZFS calls it, on the fly. Having the ability to tune the file system without recreating it saves tremendous amounts of downtime and headache. A great FileBench comparison between Veritas and ZFS, done by a Sun engineer, shows which areas ZFS excels the most in. ZFS also supports compression, which dramatically increases performance. Even on a slower machine by today’s standards, ZFS with compression on still provides much faster data access for most uses. The slow part in reading or writing to disks is in waiting for the slow heads and spindles, so reading and writing less (because it’s compressed) is very helpful. Don’t forget that compression can sometimes double your amount of space, too. Benchmarks agree. James Dickens blogged about his findings while installing Solaris Zones on UFS, ZFS, and ZFS with compression. Managing Storage Dominic Kay, an engineer from Sun, wrote about the differences in managing ZFS versus VxVM/FS. Take a look at the second table that shows the error-prone list of commands required to set up Vertias storage compared to the two ZFS commands. Of course storage administrators still need to understand every detail of what they’re doing, but the key difference here is that the ZFS file systems are created instantly, whereas the Veritas ones can take hours to initialize. Unfortunately, the concept of “managing storage” also needs to take into account the previously mentioned interoperability aspects. Veritas is stable and available on all platforms; ZFS is not. ZFS is open source, and people are working on porting it to other operating systems, but the going is slow. If you can get away with all-Solaris, and know that you’ll never want to move these file systems to another platform, ZFS is the clear winner. Annoyances The number one ZFS annoyance is that it’s only available for Solaris right now. That’s not a limitation; the fact that we want it everywhere attests to ZFS’s greatness. Another frequent complaint is that ZFS can’t live on the root file system. People would like to use ZFS instead of Solaris’s SVM to mirror operating system disks. This is being taken care of, but its usefulness is debatable. Certainly when ZFS is ported to other operating systems, they won’t want or need to support having a ZFS root file system. Many believe that the more time-tested file systems and volume managers should handle the OS’s file system. A big point of confusion for new ZFS adopters is backups. There’s no ‘dump’ command for ZFS, they expect you to use snapshots and ‘zfs send.’ The ZFS ‘send’ command allows administrators to send a file system wherever they wish: tape, a file, wherever. Traditionally, backups are done incrementally; backup everything that changes over a period of time, and every once in a while backup the entire file system as well. The problem is that you cannot perform incremental backups with traditional tools, because there’s no zfsdump equivalent. A ZFS RAIDz pool will survive if a disk dies, so Sun is trying to change how people backup data. The idea is to use snapshots daily, hourly, or however often you need. A snapshot stores only things that change since the snapshot, using very little space, so that’s your incremental backup—online and ready for access. Daily or weekly full dumps of a file system can still be performed for disaster recovery purposes. This is all well and good—most people can handle the fact that incremental backups are stored in snapshots, since the ZFS pool can be made impervious to failures—but there are drawbacks. ZFS does not support quotas. It does, but the notion of “quota” in ZFS means that you’re limiting the size of the file system. The intended design is that each user or project will get their own file system, and the appropriate quotas get applied. ZFS handles thousands of file systems very well, so this isn’t a problem, until we start thinking about backups again. ZFS snapshots can only be stored on the file system in question, in a .zfs directory. If there are active snapshots, all changes get copied into them. So when users come close to their quota, and want to delete files to free up space, they cannot. The data just gets moved into the snapshot, so an administrator needs to free their space by removing snapshots. The ability to store snapshots at a higher level, or on another file system would solve this problem. Sun hasn’t announced any plans to fix this though. The final annoyance is that you cannot remove devices from pools. You can replace them if one happens to fail, but the pool itself will not allow you to decrease its size. Sun is working on this problem. Minor annoyances aside, ZFS is great. In Solaris environments there’s no reason not to use ZFS file systems. In mixed environments, where you’d like the portability Veritas products provides, ZFS will likely be viable in another few years. References:
ZFS best practices from the Solaris Internals authors
ZFS administration guide |