Shared data storage between Windows XP/7 and Ubuntu using NTFS
This issue was a pain in the ass from the beginning and it still is to the present day. Linux and Windows handle files and their permissions differently which leads to unavoidable complications. For instance Linux sticks to the old UNIX owner-group-world permissions style while Windows makes use of access control lists. As far as I know there is no way to have both within a single file system. Another annoying thing is the way the systems handle hidden files differently. Some of these issues can be circumvented, some you just have to live with. And in order to make people use free and open-source software I made up this tutorial.
I will give you instructions on how to set up Linux and Windows side-by-side using the same data storage, meaning that the Linux home folder will be the same as the Windows My Documents folder (user profile folder on 7 respectively) and both operating systems will share the same desktop items. Therefore we will make use of a NTFS partition to store the data.
Why NTFS?
First we have to pick a filesystem both operating systems can read. The native FS on Linux is Ext2/3/4 for the time being. There are two widely spread Ext2 drivers for Windows wich are compatible to Ext3 as well since Ext3 is just a journaled Ext2. One is Ext2IFS and the second is Ext2fsd. Both can be considered beta quality and therefore in my opinion not suitable for holding my precious porn collection. I’ve read about bad experiences including severe data loss after switching from XP to 7.
On the other hand we are provided with the fine NTFS-3G driver for Linux which can be considered stable and is even equipped with partial journaling capabilities. So to say a driver I would recommend to my grandma by now. The major drawback in this solution is that access permissions for a NTFS partition under Linux can only be set globaly for all files at mount time and are used on reading and writing. Therefore this solution is not recommended for any kind of multiuser environment! In my case the only risk is my roommate spying on me, a fear that follows me like a cold, dark shadow all the time.
[UPDATE 10/11/10] Some people reported issues with the pulseaudio sound server. It seams this is related with the fact that pulseaudio repeatedly tries to check permissions of the .pulse folder in home. The NTFS-3G driver produces a special case where this check fails and therefore traps pulseaudio in a loop. Read this bug report for a more detailed description. This is not the drivers fault but rather a design flaw in pulseaudio. So this issue is likely to be fixed in a future release.
I assume you are able to set up your partitions and operating systems to meet your requirements. I used two rather small partitions for the systems (NTFS and Ext4, á 100 GiB) a swap partition of appropriate size and a big NTFS partition for my data.
Windows 7 profile folder
If you are using Windows XP just skip this chapter. Windows 7 is the pickiest system when it comes to changing location and structure of your profile folder so we will start with that. The easiest and safest way I found to do this is to first create a dummy user with administrator rights. After logging on using the dummy user open up a registry editor by clicking on the Windows button and entering ‘regedit’ into the search field on the bottom (This program is located in C:\Windows\System32\regedt32.exe). Navigate to
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\ProfileList
Change the Key ‘ProfilesDirectory’ to the new data partition (i.e. ‘D:\’ in my case). Do not use a ‘Users’ subfolder since Linux does not as well! The trick is that Windows will put profile folders of new users in this partition from now on. So this is the time to set up your actual user account in the control panel and logon afterwards. 7 will now create a new folder that has the name of your user account in the data partition. Sadly Windows dumps a shitload of files and folders into the profile folder which are hidden on Windows but you cannot hide on Linux.
Another good thing is to rename certain document folders that begin with ‘My …’. Ubuntu uses the same standard names for these folders without the prefix. Rename ‘My Pictures’ to ‘Pictures’, ‘My Videos’ to ‘Videos’ and so forth.
Windows XP My Documents folder
The process in Windows XP is quite easy. Just create a folder on the new data partition and name it after your user account. Then open up the preferences of the ‘My Documents’ folder by right-clicking on it. Within the preferences you can just alter the location of the folder. It will then ask you if you would like to move your stuff to the new location. Like in 7 it is a good idea to rename folders that start with ‘My …’ since Linux uses the same Folders without the prefix. Rename ‘My Pictures’ to ‘Pictures’, ‘My Music’ to ‘Music’ and so forth.
While 7 moved the Desktop folder into the user’s profile folder (like Linux), XP still uses a folder one level above. We have to change this. The weapon of choice is TweakUI which can be obtained on the Microsoft page. Create a folder called ‘Desktop’ directly under the new ‘My Documents’ folder. Start TweakUI and navigate to ‘My Computer/Special Folders’. Use the interface to change the location of the desktop folder. Now you can be sure Windows and Ubuntu show the same items on the desktop.
Linux home folder
Back in Ubuntu (or any other distribution of your choice) we have to mount the NTFS data partition in the home directory. Therefore we add a line to the fstab in order to mount the partition during startup. Open up a terminal window and type:
sudo gedit /etc/fstab
Scroll to the end and append the following line
UUID=30E6BE74E6BE39C4 /home ntfs-3g nodev,nosuid,gid=1000,uid=1000,umask=022 0 0
The UUID is a unique identifier for devices and partitions that is used on modern Linux systems instead of the device nodes which can be subject of change. You have to change this value to your partition’s UUID. On Ubuntu systems you can associate UUIDs with device nodes by typing into a terminal
ls -l /dev/disk/by-uuid
Also tools as GParted (available in almost every software repository) would tell you the UUID of partitions.
I chose ‘/home’ as mount point which means that all home folders on my system will be on that partition. Instead one could choose ‘/home/user’ as mount point to use the data partition only for one user but would have to take that into account in the chapters above.
The third parameter specifies the filesystem type. ‘ntfs-3g’ works just fine, assuming of course you have the NTFS-3G driver installed. If not, do so. Recent Ubuntu systems come with NTFS-3G pre-installed. The next column is a little trickier.
nodev
Prevents users to mount devices in the filesystem
nosuid
Prevents set-user-identifier or set-group-identifier bits to take effect
gid/uid
These two options define the group and the owner of ALL files within the filesystem. The main user on Ubuntu systems usually has the gid/uid 1000. Adjust these parameters as you need.
umask
Sets the actual permissions for every file within the filesystem. ’022′ is equivalent to rwxr-xr-x. Keep in mind, since this is a bit mask the syntax differs from the chmod syntax you might be used to.
The last two zeros indicate that the partition is never dumped (ignore this) and that the filesystem will never be checked. To my knowledge there is yet no substitution for Windows’ Checkdisk. So consider to check your NTFS partition on Windows from time to time. You should also keep in mind that Linux allows characters in filenames which are prohibited in Windows (” * / : < > ? \ |). Checkdisk will delete those files!!! Recent NTFS-3G drivers introduce the mount option ‘windows_names’ which restricts the character set for new filenames to Windows standards. This can have unexpected side-effects though. The GVFS (Gnome Virtual File System) for instance writes filenames with colons into ‘~/.local/share/gvfs-metadata’ if you access network resources in Nautilus. I don’t know whether this is a problem or not. I just left the mount option out without any troubles so far.
For more information on fstab syntax check out the Wikipedia page or the manual page in a terminal window:
man fstab
Linux should now mount the NTFS partition on startup. If the usernames in both of your systems are the same you are done at this point (mind: Linux is case sensitive!!). If not, the home folder name on the data partition differs from the username and therefore you have to tell Linux where to find the home folder. Open up a terminal and type
sudo gedit /etc/passwd
Find the line that starts with your username and change the home folder location from ‘/home/<username>’ to the correct location.
Hooray, you’re done!
Further things
I told you about the files and folders Windows 7 stores in the profile folder which are for the system, not for the user. Well Linux applications do the same thing. These files and folders usually are located directly in the home folder and start with a dot. Thereby they are hidden in Linux. In Windows you can hide them by checking the ‘hidden’ box in the files/folders’ preferences panel.
Recent versions of the NTFS-3G driver (I think 2010.8.8 and newer) introduce two options called ‘hide_hid_files’ and ‘hide_dot_files’. The former prevents files and folders with a set hidden flag from being listed in directory listings. That would mean that they do not appear in File Browsers for instance. The latter causes the hidden flag to be set whenever a file or folder starts with a dot. I didn’t have the chance to test it yet but it looks promising. These options would go in the fstab file next to the gid/uid/umask stuff. Feel free to run a test trial and write a comment.