2007-12-27

UDF, the new lingua franca of filesystems?

Not too long ago, when creating a filesystem on a removable storage device, or on a hard disk shared between several different operating systems, the natural choice was FAT32. FAT32 is reasonably well supported on all major OSes, however its limitations (eg. 4G maximum file size) make it unsuitable for large media.

So what is an adequate replacement for FAT32? NTFS support in Linux has improved a lot in the last years. Unfortunately, it is based on reverse engineering instead of public specifications. Due to its proprietary nature, the NTFS specification can change without notice, possibly leading to data loss/corruption when accessed with unprepared tools.

Enter UDF, which was developed for removable optical media, has no silly limitations and can be read and written to by current versions of Linux, Windows and Mac OS X. One minor issue is that the UDF specification does not allow placing it inside a partition, it must always occupy a whole disk. But no ill effects are expected from ignoring this.

Now in Linux create a partition of type 0x7 (IFS), run mkudffs, and boot into Windows. Whoops, no filesystem recognized. Ok then, run format /fs:udf then boot into Linux again. Whoops, UDF-fs says "No partition found". Experiment with different UDF revisions, no luck. Examine more closely, find out that the Linux UDF driver defaults to a block size of 2048 bytes, while Windows format uses 512 byte blocks. Mount with mount -o bs=512 and it works! Well, almost.

Copy a small file on the partition and it is recognized by the other OS. Copy a 6G file with Linux, and Windows doesn't see it. What is going on here?