Loading...
1The Linux NTFS filesystem driver
2================================
3
4
5Table of contents
6=================
7
8- Overview
9- Web site
10- Features
11- Supported mount options
12- Known bugs and (mis-)features
13- Using NTFS volume and stripe sets
14 - The Device-Mapper driver
15 - The Software RAID / MD driver
16 - Limitations when using the MD driver
17- ChangeLog
18
19
20Overview
21========
22
23Linux-NTFS comes with a number of user-space programs known as ntfsprogs.
24These include mkntfs, a full-featured ntfs filesystem format utility,
25ntfsundelete used for recovering files that were unintentionally deleted
26from an NTFS volume and ntfsresize which is used to resize an NTFS partition.
27See the web site for more information.
28
29To mount an NTFS 1.2/3.x (Windows NT4/2000/XP/2003) volume, use the file
30system type 'ntfs'. The driver currently supports read-only mode (with no
31fault-tolerance, encryption or journalling) and very limited, but safe, write
32support.
33
34For fault tolerance and raid support (i.e. volume and stripe sets), you can
35use the kernel's Software RAID / MD driver. See section "Using Software RAID
36with NTFS" for details.
37
38
39Web site
40========
41
42There is plenty of additional information on the linux-ntfs web site
43at http://www.linux-ntfs.org/
44
45The web site has a lot of additional information, such as a comprehensive
46FAQ, documentation on the NTFS on-disk format, information on the Linux-NTFS
47userspace utilities, etc.
48
49
50Features
51========
52
53- This is a complete rewrite of the NTFS driver that used to be in the 2.4 and
54 earlier kernels. This new driver implements NTFS read support and is
55 functionally equivalent to the old ntfs driver and it also implements limited
56 write support. The biggest limitation at present is that files/directories
57 cannot be created or deleted. See below for the list of write features that
58 are so far supported. Another limitation is that writing to compressed files
59 is not implemented at all. Also, neither read nor write access to encrypted
60 files is so far implemented.
61- The new driver has full support for sparse files on NTFS 3.x volumes which
62 the old driver isn't happy with.
63- The new driver supports execution of binaries due to mmap() now being
64 supported.
65- The new driver supports loopback mounting of files on NTFS which is used by
66 some Linux distributions to enable the user to run Linux from an NTFS
67 partition by creating a large file while in Windows and then loopback
68 mounting the file while in Linux and creating a Linux filesystem on it that
69 is used to install Linux on it.
70- A comparison of the two drivers using:
71 time find . -type f -exec md5sum "{}" \;
72 run three times in sequence with each driver (after a reboot) on a 1.4GiB
73 NTFS partition, showed the new driver to be 20% faster in total time elapsed
74 (from 9:43 minutes on average down to 7:53). The time spent in user space
75 was unchanged but the time spent in the kernel was decreased by a factor of
76 2.5 (from 85 CPU seconds down to 33).
77- The driver does not support short file names in general. For backwards
78 compatibility, we implement access to files using their short file names if
79 they exist. The driver will not create short file names however, and a
80 rename will discard any existing short file name.
81- The new driver supports exporting of mounted NTFS volumes via NFS.
82- The new driver supports async io (aio).
83- The new driver supports fsync(2), fdatasync(2), and msync(2).
84- The new driver supports readv(2) and writev(2).
85- The new driver supports access time updates (including mtime and ctime).
86- The new driver supports truncate(2) and open(2) with O_TRUNC. But at present
87 only very limited support for highly fragmented files, i.e. ones which have
88 their data attribute split across multiple extents, is included. Another
89 limitation is that at present truncate(2) will never create sparse files,
90 since to mark a file sparse we need to modify the directory entry for the
91 file and we do not implement directory modifications yet.
92- The new driver supports write(2) which can both overwrite existing data and
93 extend the file size so that you can write beyond the existing data. Also,
94 writing into sparse regions is supported and the holes are filled in with
95 clusters. But at present only limited support for highly fragmented files,
96 i.e. ones which have their data attribute split across multiple extents, is
97 included. Another limitation is that write(2) will never create sparse
98 files, since to mark a file sparse we need to modify the directory entry for
99 the file and we do not implement directory modifications yet.
100
101Supported mount options
102=======================
103
104In addition to the generic mount options described by the manual page for the
105mount command (man 8 mount, also see man 5 fstab), the NTFS driver supports the
106following mount options:
107
108iocharset=name Deprecated option. Still supported but please use
109 nls=name in the future. See description for nls=name.
110
111nls=name Character set to use when returning file names.
112 Unlike VFAT, NTFS suppresses names that contain
113 unconvertible characters. Note that most character
114 sets contain insufficient characters to represent all
115 possible Unicode characters that can exist on NTFS.
116 To be sure you are not missing any files, you are
117 advised to use nls=utf8 which is capable of
118 representing all Unicode characters.
119
120utf8=<bool> Option no longer supported. Currently mapped to
121 nls=utf8 but please use nls=utf8 in the future and
122 make sure utf8 is compiled either as module or into
123 the kernel. See description for nls=name.
124
125uid=
126gid=
127umask= Provide default owner, group, and access mode mask.
128 These options work as documented in mount(8). By
129 default, the files/directories are owned by root and
130 he/she has read and write permissions, as well as
131 browse permission for directories. No one else has any
132 access permissions. I.e. the mode on all files is by
133 default rw------- and for directories rwx------, a
134 consequence of the default fmask=0177 and dmask=0077.
135 Using a umask of zero will grant all permissions to
136 everyone, i.e. all files and directories will have mode
137 rwxrwxrwx.
138
139fmask=
140dmask= Instead of specifying umask which applies both to
141 files and directories, fmask applies only to files and
142 dmask only to directories.
143
144sloppy=<BOOL> If sloppy is specified, ignore unknown mount options.
145 Otherwise the default behaviour is to abort mount if
146 any unknown options are found.
147
148show_sys_files=<BOOL> If show_sys_files is specified, show the system files
149 in directory listings. Otherwise the default behaviour
150 is to hide the system files.
151 Note that even when show_sys_files is specified, "$MFT"
152 will not be visible due to bugs/mis-features in glibc.
153 Further, note that irrespective of show_sys_files, all
154 files are accessible by name, i.e. you can always do
155 "ls -l \$UpCase" for example to specifically show the
156 system file containing the Unicode upcase table.
157
158case_sensitive=<BOOL> If case_sensitive is specified, treat all file names as
159 case sensitive and create file names in the POSIX
160 namespace. Otherwise the default behaviour is to treat
161 file names as case insensitive and to create file names
162 in the WIN32/LONG name space. Note, the Linux NTFS
163 driver will never create short file names and will
164 remove them on rename/delete of the corresponding long
165 file name.
166 Note that files remain accessible via their short file
167 name, if it exists. If case_sensitive, you will need
168 to provide the correct case of the short file name.
169
170disable_sparse=<BOOL> If disable_sparse is specified, creation of sparse
171 regions, i.e. holes, inside files is disabled for the
172 volume (for the duration of this mount only). By
173 default, creation of sparse regions is enabled, which
174 is consistent with the behaviour of traditional Unix
175 filesystems.
176
177errors=opt What to do when critical filesystem errors are found.
178 Following values can be used for "opt":
179 continue: DEFAULT, try to clean-up as much as
180 possible, e.g. marking a corrupt inode as
181 bad so it is no longer accessed, and then
182 continue.
183 recover: At present only supported is recovery of
184 the boot sector from the backup copy.
185 If read-only mount, the recovery is done
186 in memory only and not written to disk.
187 Note that the options are additive, i.e. specifying:
188 errors=continue,errors=recover
189 means the driver will attempt to recover and if that
190 fails it will clean-up as much as possible and
191 continue.
192
193mft_zone_multiplier= Set the MFT zone multiplier for the volume (this
194 setting is not persistent across mounts and can be
195 changed from mount to mount but cannot be changed on
196 remount). Values of 1 to 4 are allowed, 1 being the
197 default. The MFT zone multiplier determines how much
198 space is reserved for the MFT on the volume. If all
199 other space is used up, then the MFT zone will be
200 shrunk dynamically, so this has no impact on the
201 amount of free space. However, it can have an impact
202 on performance by affecting fragmentation of the MFT.
203 In general use the default. If you have a lot of small
204 files then use a higher value. The values have the
205 following meaning:
206 Value MFT zone size (% of volume size)
207 1 12.5%
208 2 25%
209 3 37.5%
210 4 50%
211 Note this option is irrelevant for read-only mounts.
212
213
214Known bugs and (mis-)features
215=============================
216
217- The link count on each directory inode entry is set to 1, due to Linux not
218 supporting directory hard links. This may well confuse some user space
219 applications, since the directory names will have the same inode numbers.
220 This also speeds up ntfs_read_inode() immensely. And we haven't found any
221 problems with this approach so far. If you find a problem with this, please
222 let us know.
223
224
225Please send bug reports/comments/feedback/abuse to the Linux-NTFS development
226list at sourceforge: linux-ntfs-dev@lists.sourceforge.net
227
228
229Using NTFS volume and stripe sets
230=================================
231
232For support of volume and stripe sets, you can either use the kernel's
233Device-Mapper driver or the kernel's Software RAID / MD driver. The former is
234the recommended one to use for linear raid. But the latter is required for
235raid level 5. For striping and mirroring, either driver should work fine.
236
237
238The Device-Mapper driver
239------------------------
240
241You will need to create a table of the components of the volume/stripe set and
242how they fit together and load this into the kernel using the dmsetup utility
243(see man 8 dmsetup).
244
245Linear volume sets, i.e. linear raid, has been tested and works fine. Even
246though untested, there is no reason why stripe sets, i.e. raid level 0, and
247mirrors, i.e. raid level 1 should not work, too. Stripes with parity, i.e.
248raid level 5, unfortunately cannot work yet because the current version of the
249Device-Mapper driver does not support raid level 5. You may be able to use the
250Software RAID / MD driver for raid level 5, see the next section for details.
251
252To create the table describing your volume you will need to know each of its
253components and their sizes in sectors, i.e. multiples of 512-byte blocks.
254
255For NT4 fault tolerant volumes you can obtain the sizes using fdisk. So for
256example if one of your partitions is /dev/hda2 you would do:
257
258$ fdisk -ul /dev/hda
259
260Disk /dev/hda: 81.9 GB, 81964302336 bytes
261255 heads, 63 sectors/track, 9964 cylinders, total 160086528 sectors
262Units = sectors of 1 * 512 = 512 bytes
263
264 Device Boot Start End Blocks Id System
265 /dev/hda1 * 63 4209029 2104483+ 83 Linux
266 /dev/hda2 4209030 37768814 16779892+ 86 NTFS
267 /dev/hda3 37768815 46170809 4200997+ 83 Linux
268
269And you would know that /dev/hda2 has a size of 37768814 - 4209030 + 1 =
27033559785 sectors.
271
272For Win2k and later dynamic disks, you can for example use the ldminfo utility
273which is part of the Linux LDM tools (the latest version at the time of
274writing is linux-ldm-0.0.8.tar.bz2). You can download it from:
275 http://www.linux-ntfs.org/
276Simply extract the downloaded archive (tar xvjf linux-ldm-0.0.8.tar.bz2), go
277into it (cd linux-ldm-0.0.8) and change to the test directory (cd test). You
278will find the precompiled (i386) ldminfo utility there. NOTE: You will not be
279able to compile this yourself easily so use the binary version!
280
281Then you would use ldminfo in dump mode to obtain the necessary information:
282
283$ ./ldminfo --dump /dev/hda
284
285This would dump the LDM database found on /dev/hda which describes all of your
286dynamic disks and all the volumes on them. At the bottom you will see the
287VOLUME DEFINITIONS section which is all you really need. You may need to look
288further above to determine which of the disks in the volume definitions is
289which device in Linux. Hint: Run ldminfo on each of your dynamic disks and
290look at the Disk Id close to the top of the output for each (the PRIVATE HEADER
291section). You can then find these Disk Ids in the VBLK DATABASE section in the
292<Disk> components where you will get the LDM Name for the disk that is found in
293the VOLUME DEFINITIONS section.
294
295Note you will also need to enable the LDM driver in the Linux kernel. If your
296distribution did not enable it, you will need to recompile the kernel with it
297enabled. This will create the LDM partitions on each device at boot time. You
298would then use those devices (for /dev/hda they would be /dev/hda1, 2, 3, etc)
299in the Device-Mapper table.
300
301You can also bypass using the LDM driver by using the main device (e.g.
302/dev/hda) and then using the offsets of the LDM partitions into this device as
303the "Start sector of device" when creating the table. Once again ldminfo would
304give you the correct information to do this.
305
306Assuming you know all your devices and their sizes things are easy.
307
308For a linear raid the table would look like this (note all values are in
309512-byte sectors):
310
311--- cut here ---
312# Offset into Size of this Raid type Device Start sector
313# volume device of device
3140 1028161 linear /dev/hda1 0
3151028161 3903762 linear /dev/hdb2 0
3164931923 2103211 linear /dev/hdc1 0
317--- cut here ---
318
319For a striped volume, i.e. raid level 0, you will need to know the chunk size
320you used when creating the volume. Windows uses 64kiB as the default, so it
321will probably be this unless you changes the defaults when creating the array.
322
323For a raid level 0 the table would look like this (note all values are in
324512-byte sectors):
325
326--- cut here ---
327# Offset Size Raid Number Chunk 1st Start 2nd Start
328# into of the type of size Device in Device in
329# volume volume stripes device device
3300 2056320 striped 2 128 /dev/hda1 0 /dev/hdb1 0
331--- cut here ---
332
333If there are more than two devices, just add each of them to the end of the
334line.
335
336Finally, for a mirrored volume, i.e. raid level 1, the table would look like
337this (note all values are in 512-byte sectors):
338
339--- cut here ---
340# Ofs Size Raid Log Number Region Should Number Source Start Target Start
341# in of the type type of log size sync? of Device in Device in
342# vol volume params mirrors Device Device
3430 2056320 mirror core 2 16 nosync 2 /dev/hda1 0 /dev/hdb1 0
344--- cut here ---
345
346If you are mirroring to multiple devices you can specify further targets at the
347end of the line.
348
349Note the "Should sync?" parameter "nosync" means that the two mirrors are
350already in sync which will be the case on a clean shutdown of Windows. If the
351mirrors are not clean, you can specify the "sync" option instead of "nosync"
352and the Device-Mapper driver will then copy the entirety of the "Source Device"
353to the "Target Device" or if you specified multiple target devices to all of
354them.
355
356Once you have your table, save it in a file somewhere (e.g. /etc/ntfsvolume1),
357and hand it over to dmsetup to work with, like so:
358
359$ dmsetup create myvolume1 /etc/ntfsvolume1
360
361You can obviously replace "myvolume1" with whatever name you like.
362
363If it all worked, you will now have the device /dev/device-mapper/myvolume1
364which you can then just use as an argument to the mount command as usual to
365mount the ntfs volume. For example:
366
367$ mount -t ntfs -o ro /dev/device-mapper/myvolume1 /mnt/myvol1
368
369(You need to create the directory /mnt/myvol1 first and of course you can use
370anything you like instead of /mnt/myvol1 as long as it is an existing
371directory.)
372
373It is advisable to do the mount read-only to see if the volume has been setup
374correctly to avoid the possibility of causing damage to the data on the ntfs
375volume.
376
377
378The Software RAID / MD driver
379-----------------------------
380
381An alternative to using the Device-Mapper driver is to use the kernel's
382Software RAID / MD driver. For which you need to set up your /etc/raidtab
383appropriately (see man 5 raidtab).
384
385Linear volume sets, i.e. linear raid, as well as stripe sets, i.e. raid level
3860, have been tested and work fine (though see section "Limitations when using
387the MD driver with NTFS volumes" especially if you want to use linear raid).
388Even though untested, there is no reason why mirrors, i.e. raid level 1, and
389stripes with parity, i.e. raid level 5, should not work, too.
390
391You have to use the "persistent-superblock 0" option for each raid-disk in the
392NTFS volume/stripe you are configuring in /etc/raidtab as the persistent
393superblock used by the MD driver would damage the NTFS volume.
394
395Windows by default uses a stripe chunk size of 64k, so you probably want the
396"chunk-size 64k" option for each raid-disk, too.
397
398For example, if you have a stripe set consisting of two partitions /dev/hda5
399and /dev/hdb1 your /etc/raidtab would look like this:
400
401raiddev /dev/md0
402 raid-level 0
403 nr-raid-disks 2
404 nr-spare-disks 0
405 persistent-superblock 0
406 chunk-size 64k
407 device /dev/hda5
408 raid-disk 0
409 device /dev/hdb1
410 raid-disk 1
411
412For linear raid, just change the raid-level above to "raid-level linear", for
413mirrors, change it to "raid-level 1", and for stripe sets with parity, change
414it to "raid-level 5".
415
416Note for stripe sets with parity you will also need to tell the MD driver
417which parity algorithm to use by specifying the option "parity-algorithm
418which", where you need to replace "which" with the name of the algorithm to
419use (see man 5 raidtab for available algorithms) and you will have to try the
420different available algorithms until you find one that works. Make sure you
421are working read-only when playing with this as you may damage your data
422otherwise. If you find which algorithm works please let us know (email the
423linux-ntfs developers list linux-ntfs-dev@lists.sourceforge.net or drop in on
424IRC in channel #ntfs on the irc.freenode.net network) so we can update this
425documentation.
426
427Once the raidtab is setup, run for example raid0run -a to start all devices or
428raid0run /dev/md0 to start a particular md device, in this case /dev/md0.
429
430Then just use the mount command as usual to mount the ntfs volume using for
431example: mount -t ntfs -o ro /dev/md0 /mnt/myntfsvolume
432
433It is advisable to do the mount read-only to see if the md volume has been
434setup correctly to avoid the possibility of causing damage to the data on the
435ntfs volume.
436
437
438Limitations when using the Software RAID / MD driver
439-----------------------------------------------------
440
441Using the md driver will not work properly if any of your NTFS partitions have
442an odd number of sectors. This is especially important for linear raid as all
443data after the first partition with an odd number of sectors will be offset by
444one or more sectors so if you mount such a partition with write support you
445will cause massive damage to the data on the volume which will only become
446apparent when you try to use the volume again under Windows.
447
448So when using linear raid, make sure that all your partitions have an even
449number of sectors BEFORE attempting to use it. You have been warned!
450
451Even better is to simply use the Device-Mapper for linear raid and then you do
452not have this problem with odd numbers of sectors.
453
454
455ChangeLog
456=========
457
458Note, a technical ChangeLog aimed at kernel hackers is in fs/ntfs/ChangeLog.
459
4602.1.30:
461 - Fix writev() (it kept writing the first segment over and over again
462 instead of moving onto subsequent segments).
463 - Fix crash in ntfs_mft_record_alloc() when mapping the new extent mft
464 record failed.
4652.1.29:
466 - Fix a deadlock when mounting read-write.
4672.1.28:
468 - Fix a deadlock.
4692.1.27:
470 - Implement page migration support so the kernel can move memory used
471 by NTFS files and directories around for management purposes.
472 - Add support for writing to sparse files created with Windows XP SP2.
473 - Many minor improvements and bug fixes.
4742.1.26:
475 - Implement support for sector sizes above 512 bytes (up to the maximum
476 supported by NTFS which is 4096 bytes).
477 - Enhance support for NTFS volumes which were supported by Windows but
478 not by Linux due to invalid attribute list attribute flags.
479 - A few minor updates and bug fixes.
4802.1.25:
481 - Write support is now extended with write(2) being able to both
482 overwrite existing file data and to extend files. Also, if a write
483 to a sparse region occurs, write(2) will fill in the hole. Note,
484 mmap(2) based writes still do not support writing into holes or
485 writing beyond the initialized size.
486 - Write support has a new feature and that is that truncate(2) and
487 open(2) with O_TRUNC are now implemented thus files can be both made
488 smaller and larger.
489 - Note: Both write(2) and truncate(2)/open(2) with O_TRUNC still have
490 limitations in that they
491 - only provide limited support for highly fragmented files.
492 - only work on regular, i.e. uncompressed and unencrypted files.
493 - never create sparse files although this will change once directory
494 operations are implemented.
495 - Lots of bug fixes and enhancements across the board.
4962.1.24:
497 - Support journals ($LogFile) which have been modified by chkdsk. This
498 means users can boot into Windows after we marked the volume dirty.
499 The Windows boot will run chkdsk and then reboot. The user can then
500 immediately boot into Linux rather than having to do a full Windows
501 boot first before rebooting into Linux and we will recognize such a
502 journal and empty it as it is clean by definition.
503 - Support journals ($LogFile) with only one restart page as well as
504 journals with two different restart pages. We sanity check both and
505 either use the only sane one or the more recent one of the two in the
506 case that both are valid.
507 - Lots of bug fixes and enhancements across the board.
5082.1.23:
509 - Stamp the user space journal, aka transaction log, aka $UsnJrnl, if
510 it is present and active thus telling Windows and applications using
511 the transaction log that changes can have happened on the volume
512 which are not recorded in $UsnJrnl.
513 - Detect the case when Windows has been hibernated (suspended to disk)
514 and if this is the case do not allow (re)mounting read-write to
515 prevent data corruption when you boot back into the suspended
516 Windows session.
517 - Implement extension of resident files using the normal file write
518 code paths, i.e. most very small files can be extended to be a little
519 bit bigger but not by much.
520 - Add new mount option "disable_sparse". (See list of mount options
521 above for details.)
522 - Improve handling of ntfs volumes with errors and strange boot sectors
523 in particular.
524 - Fix various bugs including a nasty deadlock that appeared in recent
525 kernels (around 2.6.11-2.6.12 timeframe).
5262.1.22:
527 - Improve handling of ntfs volumes with errors.
528 - Fix various bugs and race conditions.
5292.1.21:
530 - Fix several race conditions and various other bugs.
531 - Many internal cleanups, code reorganization, optimizations, and mft
532 and index record writing code rewritten to fit in with the changes.
533 - Update Documentation/filesystems/ntfs.txt with instructions on how to
534 use the Device-Mapper driver with NTFS ftdisk/LDM raid.
5352.1.20:
536 - Fix two stupid bugs introduced in 2.1.18 release.
5372.1.19:
538 - Minor bugfix in handling of the default upcase table.
539 - Many internal cleanups and improvements. Many thanks to Linus
540 Torvalds and Al Viro for the help and advice with the sparse
541 annotations and cleanups.
5422.1.18:
543 - Fix scheduling latencies at mount time. (Ingo Molnar)
544 - Fix endianness bug in a little traversed portion of the attribute
545 lookup code.
5462.1.17:
547 - Fix bugs in mount time error code paths.
5482.1.16:
549 - Implement access time updates (including mtime and ctime).
550 - Implement fsync(2), fdatasync(2), and msync(2) system calls.
551 - Enable the readv(2) and writev(2) system calls.
552 - Enable access via the asynchronous io (aio) API by adding support for
553 the aio_read(3) and aio_write(3) functions.
5542.1.15:
555 - Invalidate quotas when (re)mounting read-write.
556 NOTE: This now only leave user space journalling on the side. (See
557 note for version 2.1.13, below.)
5582.1.14:
559 - Fix an NFSd caused deadlock reported by several users.
5602.1.13:
561 - Implement writing of inodes (access time updates are not implemented
562 yet so mounting with -o noatime,nodiratime is enforced).
563 - Enable writing out of resident files so you can now overwrite any
564 uncompressed, unencrypted, nonsparse file as long as you do not
565 change the file size.
566 - Add housekeeping of ntfs system files so that ntfsfix no longer needs
567 to be run after writing to an NTFS volume.
568 NOTE: This still leaves quota tracking and user space journalling on
569 the side but they should not cause data corruption. In the worst
570 case the charged quotas will be out of date ($Quota) and some
571 userspace applications might get confused due to the out of date
572 userspace journal ($UsnJrnl).
5732.1.12:
574 - Fix the second fix to the decompression engine from the 2.1.9 release
575 and some further internals cleanups.
5762.1.11:
577 - Driver internal cleanups.
5782.1.10:
579 - Force read-only (re)mounting of volumes with unsupported volume
580 flags and various cleanups.
5812.1.9:
582 - Fix two bugs in handling of corner cases in the decompression engine.
5832.1.8:
584 - Read the $MFT mirror and compare it to the $MFT and if the two do not
585 match, force a read-only mount and do not allow read-write remounts.
586 - Read and parse the $LogFile journal and if it indicates that the
587 volume was not shutdown cleanly, force a read-only mount and do not
588 allow read-write remounts. If the $LogFile indicates a clean
589 shutdown and a read-write (re)mount is requested, empty $LogFile to
590 ensure that Windows cannot cause data corruption by replaying a stale
591 journal after Linux has written to the volume.
592 - Improve time handling so that the NTFS time is fully preserved when
593 converted to kernel time and only up to 99 nano-seconds are lost when
594 kernel time is converted to NTFS time.
5952.1.7:
596 - Enable NFS exporting of mounted NTFS volumes.
5972.1.6:
598 - Fix minor bug in handling of compressed directories that fixes the
599 erroneous "du" and "stat" output people reported.
6002.1.5:
601 - Minor bug fix in attribute list attribute handling that fixes the
602 I/O errors on "ls" of certain fragmented files found by at least two
603 people running Windows XP.
6042.1.4:
605 - Minor update allowing compilation with all gcc versions (well, the
606 ones the kernel can be compiled with anyway).
6072.1.3:
608 - Major bug fixes for reading files and volumes in corner cases which
609 were being hit by Windows 2k/XP users.
6102.1.2:
611 - Major bug fixes alleviating the hangs in statfs experienced by some
612 users.
6132.1.1:
614 - Update handling of compressed files so people no longer get the
615 frequently reported warning messages about initialized_size !=
616 data_size.
6172.1.0:
618 - Add configuration option for developmental write support.
619 - Initial implementation of file overwriting. (Writes to resident files
620 are not written out to disk yet, so avoid writing to files smaller
621 than about 1kiB.)
622 - Intercept/abort changes in file size as they are not implemented yet.
6232.0.25:
624 - Minor bugfixes in error code paths and small cleanups.
6252.0.24:
626 - Small internal cleanups.
627 - Support for sendfile system call. (Christoph Hellwig)
6282.0.23:
629 - Massive internal locking changes to mft record locking. Fixes
630 various race conditions and deadlocks.
631 - Fix ntfs over loopback for compressed files by adding an
632 optimization barrier. (gcc was screwing up otherwise ?)
633 Thanks go to Christoph Hellwig for pointing these two out:
634 - Remove now unused function fs/ntfs/malloc.h::vmalloc_nofs().
635 - Fix ntfs_free() for ia64 and parisc.
6362.0.22:
637 - Small internal cleanups.
6382.0.21:
639 These only affect 32-bit architectures:
640 - Check for, and refuse to mount too large volumes (maximum is 2TiB).
641 - Check for, and refuse to open too large files and directories
642 (maximum is 16TiB).
6432.0.20:
644 - Support non-resident directory index bitmaps. This means we now cope
645 with huge directories without problems.
646 - Fix a page leak that manifested itself in some cases when reading
647 directory contents.
648 - Internal cleanups.
6492.0.19:
650 - Fix race condition and improvements in block i/o interface.
651 - Optimization when reading compressed files.
6522.0.18:
653 - Fix race condition in reading of compressed files.
6542.0.17:
655 - Cleanups and optimizations.
6562.0.16:
657 - Fix stupid bug introduced in 2.0.15 in new attribute inode API.
658 - Big internal cleanup replacing the mftbmp access hacks by using the
659 new attribute inode API instead.
6602.0.15:
661 - Bug fix in parsing of remount options.
662 - Internal changes implementing attribute (fake) inodes allowing all
663 attribute i/o to go via the page cache and to use all the normal
664 vfs/mm functionality.
6652.0.14:
666 - Internal changes improving run list merging code and minor locking
667 change to not rely on BKL in ntfs_statfs().
6682.0.13:
669 - Internal changes towards using iget5_locked() in preparation for
670 fake inodes and small cleanups to ntfs_volume structure.
6712.0.12:
672 - Internal cleanups in address space operations made possible by the
673 changes introduced in the previous release.
6742.0.11:
675 - Internal updates and cleanups introducing the first step towards
676 fake inode based attribute i/o.
6772.0.10:
678 - Microsoft says that the maximum number of inodes is 2^32 - 1. Update
679 the driver accordingly to only use 32-bits to store inode numbers on
680 32-bit architectures. This improves the speed of the driver a little.
6812.0.9:
682 - Change decompression engine to use a single buffer. This should not
683 affect performance except perhaps on the most heavy i/o on SMP
684 systems when accessing multiple compressed files from multiple
685 devices simultaneously.
686 - Minor updates and cleanups.
6872.0.8:
688 - Remove now obsolete show_inodes and posix mount option(s).
689 - Restore show_sys_files mount option.
690 - Add new mount option case_sensitive, to determine if the driver
691 treats file names as case sensitive or not.
692 - Mostly drop support for short file names (for backwards compatibility
693 we only support accessing files via their short file name if one
694 exists).
695 - Fix dcache aliasing issues wrt short/long file names.
696 - Cleanups and minor fixes.
6972.0.7:
698 - Just cleanups.
6992.0.6:
700 - Major bugfix to make compatible with other kernel changes. This fixes
701 the hangs/oopses on umount.
702 - Locking cleanup in directory operations (remove BKL usage).
7032.0.5:
704 - Major buffer overflow bug fix.
705 - Minor cleanups and updates for kernel 2.5.12.
7062.0.4:
707 - Cleanups and updates for kernel 2.5.11.
7082.0.3:
709 - Small bug fixes, cleanups, and performance improvements.
7102.0.2:
711 - Use default fmask of 0177 so that files are no executable by default.
712 If you want owner executable files, just use fmask=0077.
713 - Update for kernel 2.5.9 but preserve backwards compatibility with
714 kernel 2.5.7.
715 - Minor bug fixes, cleanups, and updates.
7162.0.1:
717 - Minor updates, primarily set the executable bit by default on files
718 so they can be executed.
7192.0.0:
720 - Started ChangeLog.
721
1The Linux NTFS filesystem driver
2================================
3
4
5Table of contents
6=================
7
8- Overview
9- Web site
10- Features
11- Supported mount options
12- Known bugs and (mis-)features
13- Using NTFS volume and stripe sets
14 - The Device-Mapper driver
15 - The Software RAID / MD driver
16 - Limitations when using the MD driver
17
18
19Overview
20========
21
22Linux-NTFS comes with a number of user-space programs known as ntfsprogs.
23These include mkntfs, a full-featured ntfs filesystem format utility,
24ntfsundelete used for recovering files that were unintentionally deleted
25from an NTFS volume and ntfsresize which is used to resize an NTFS partition.
26See the web site for more information.
27
28To mount an NTFS 1.2/3.x (Windows NT4/2000/XP/2003) volume, use the file
29system type 'ntfs'. The driver currently supports read-only mode (with no
30fault-tolerance, encryption or journalling) and very limited, but safe, write
31support.
32
33For fault tolerance and raid support (i.e. volume and stripe sets), you can
34use the kernel's Software RAID / MD driver. See section "Using Software RAID
35with NTFS" for details.
36
37
38Web site
39========
40
41There is plenty of additional information on the linux-ntfs web site
42at http://www.linux-ntfs.org/
43
44The web site has a lot of additional information, such as a comprehensive
45FAQ, documentation on the NTFS on-disk format, information on the Linux-NTFS
46userspace utilities, etc.
47
48
49Features
50========
51
52- This is a complete rewrite of the NTFS driver that used to be in the 2.4 and
53 earlier kernels. This new driver implements NTFS read support and is
54 functionally equivalent to the old ntfs driver and it also implements limited
55 write support. The biggest limitation at present is that files/directories
56 cannot be created or deleted. See below for the list of write features that
57 are so far supported. Another limitation is that writing to compressed files
58 is not implemented at all. Also, neither read nor write access to encrypted
59 files is so far implemented.
60- The new driver has full support for sparse files on NTFS 3.x volumes which
61 the old driver isn't happy with.
62- The new driver supports execution of binaries due to mmap() now being
63 supported.
64- The new driver supports loopback mounting of files on NTFS which is used by
65 some Linux distributions to enable the user to run Linux from an NTFS
66 partition by creating a large file while in Windows and then loopback
67 mounting the file while in Linux and creating a Linux filesystem on it that
68 is used to install Linux on it.
69- A comparison of the two drivers using:
70 time find . -type f -exec md5sum "{}" \;
71 run three times in sequence with each driver (after a reboot) on a 1.4GiB
72 NTFS partition, showed the new driver to be 20% faster in total time elapsed
73 (from 9:43 minutes on average down to 7:53). The time spent in user space
74 was unchanged but the time spent in the kernel was decreased by a factor of
75 2.5 (from 85 CPU seconds down to 33).
76- The driver does not support short file names in general. For backwards
77 compatibility, we implement access to files using their short file names if
78 they exist. The driver will not create short file names however, and a
79 rename will discard any existing short file name.
80- The new driver supports exporting of mounted NTFS volumes via NFS.
81- The new driver supports async io (aio).
82- The new driver supports fsync(2), fdatasync(2), and msync(2).
83- The new driver supports readv(2) and writev(2).
84- The new driver supports access time updates (including mtime and ctime).
85- The new driver supports truncate(2) and open(2) with O_TRUNC. But at present
86 only very limited support for highly fragmented files, i.e. ones which have
87 their data attribute split across multiple extents, is included. Another
88 limitation is that at present truncate(2) will never create sparse files,
89 since to mark a file sparse we need to modify the directory entry for the
90 file and we do not implement directory modifications yet.
91- The new driver supports write(2) which can both overwrite existing data and
92 extend the file size so that you can write beyond the existing data. Also,
93 writing into sparse regions is supported and the holes are filled in with
94 clusters. But at present only limited support for highly fragmented files,
95 i.e. ones which have their data attribute split across multiple extents, is
96 included. Another limitation is that write(2) will never create sparse
97 files, since to mark a file sparse we need to modify the directory entry for
98 the file and we do not implement directory modifications yet.
99
100Supported mount options
101=======================
102
103In addition to the generic mount options described by the manual page for the
104mount command (man 8 mount, also see man 5 fstab), the NTFS driver supports the
105following mount options:
106
107iocharset=name Deprecated option. Still supported but please use
108 nls=name in the future. See description for nls=name.
109
110nls=name Character set to use when returning file names.
111 Unlike VFAT, NTFS suppresses names that contain
112 unconvertible characters. Note that most character
113 sets contain insufficient characters to represent all
114 possible Unicode characters that can exist on NTFS.
115 To be sure you are not missing any files, you are
116 advised to use nls=utf8 which is capable of
117 representing all Unicode characters.
118
119utf8=<bool> Option no longer supported. Currently mapped to
120 nls=utf8 but please use nls=utf8 in the future and
121 make sure utf8 is compiled either as module or into
122 the kernel. See description for nls=name.
123
124uid=
125gid=
126umask= Provide default owner, group, and access mode mask.
127 These options work as documented in mount(8). By
128 default, the files/directories are owned by root and
129 he/she has read and write permissions, as well as
130 browse permission for directories. No one else has any
131 access permissions. I.e. the mode on all files is by
132 default rw------- and for directories rwx------, a
133 consequence of the default fmask=0177 and dmask=0077.
134 Using a umask of zero will grant all permissions to
135 everyone, i.e. all files and directories will have mode
136 rwxrwxrwx.
137
138fmask=
139dmask= Instead of specifying umask which applies both to
140 files and directories, fmask applies only to files and
141 dmask only to directories.
142
143sloppy=<BOOL> If sloppy is specified, ignore unknown mount options.
144 Otherwise the default behaviour is to abort mount if
145 any unknown options are found.
146
147show_sys_files=<BOOL> If show_sys_files is specified, show the system files
148 in directory listings. Otherwise the default behaviour
149 is to hide the system files.
150 Note that even when show_sys_files is specified, "$MFT"
151 will not be visible due to bugs/mis-features in glibc.
152 Further, note that irrespective of show_sys_files, all
153 files are accessible by name, i.e. you can always do
154 "ls -l \$UpCase" for example to specifically show the
155 system file containing the Unicode upcase table.
156
157case_sensitive=<BOOL> If case_sensitive is specified, treat all file names as
158 case sensitive and create file names in the POSIX
159 namespace. Otherwise the default behaviour is to treat
160 file names as case insensitive and to create file names
161 in the WIN32/LONG name space. Note, the Linux NTFS
162 driver will never create short file names and will
163 remove them on rename/delete of the corresponding long
164 file name.
165 Note that files remain accessible via their short file
166 name, if it exists. If case_sensitive, you will need
167 to provide the correct case of the short file name.
168
169disable_sparse=<BOOL> If disable_sparse is specified, creation of sparse
170 regions, i.e. holes, inside files is disabled for the
171 volume (for the duration of this mount only). By
172 default, creation of sparse regions is enabled, which
173 is consistent with the behaviour of traditional Unix
174 filesystems.
175
176errors=opt What to do when critical filesystem errors are found.
177 Following values can be used for "opt":
178 continue: DEFAULT, try to clean-up as much as
179 possible, e.g. marking a corrupt inode as
180 bad so it is no longer accessed, and then
181 continue.
182 recover: At present only supported is recovery of
183 the boot sector from the backup copy.
184 If read-only mount, the recovery is done
185 in memory only and not written to disk.
186 Note that the options are additive, i.e. specifying:
187 errors=continue,errors=recover
188 means the driver will attempt to recover and if that
189 fails it will clean-up as much as possible and
190 continue.
191
192mft_zone_multiplier= Set the MFT zone multiplier for the volume (this
193 setting is not persistent across mounts and can be
194 changed from mount to mount but cannot be changed on
195 remount). Values of 1 to 4 are allowed, 1 being the
196 default. The MFT zone multiplier determines how much
197 space is reserved for the MFT on the volume. If all
198 other space is used up, then the MFT zone will be
199 shrunk dynamically, so this has no impact on the
200 amount of free space. However, it can have an impact
201 on performance by affecting fragmentation of the MFT.
202 In general use the default. If you have a lot of small
203 files then use a higher value. The values have the
204 following meaning:
205 Value MFT zone size (% of volume size)
206 1 12.5%
207 2 25%
208 3 37.5%
209 4 50%
210 Note this option is irrelevant for read-only mounts.
211
212
213Known bugs and (mis-)features
214=============================
215
216- The link count on each directory inode entry is set to 1, due to Linux not
217 supporting directory hard links. This may well confuse some user space
218 applications, since the directory names will have the same inode numbers.
219 This also speeds up ntfs_read_inode() immensely. And we haven't found any
220 problems with this approach so far. If you find a problem with this, please
221 let us know.
222
223
224Please send bug reports/comments/feedback/abuse to the Linux-NTFS development
225list at sourceforge: linux-ntfs-dev@lists.sourceforge.net
226
227
228Using NTFS volume and stripe sets
229=================================
230
231For support of volume and stripe sets, you can either use the kernel's
232Device-Mapper driver or the kernel's Software RAID / MD driver. The former is
233the recommended one to use for linear raid. But the latter is required for
234raid level 5. For striping and mirroring, either driver should work fine.
235
236
237The Device-Mapper driver
238------------------------
239
240You will need to create a table of the components of the volume/stripe set and
241how they fit together and load this into the kernel using the dmsetup utility
242(see man 8 dmsetup).
243
244Linear volume sets, i.e. linear raid, has been tested and works fine. Even
245though untested, there is no reason why stripe sets, i.e. raid level 0, and
246mirrors, i.e. raid level 1 should not work, too. Stripes with parity, i.e.
247raid level 5, unfortunately cannot work yet because the current version of the
248Device-Mapper driver does not support raid level 5. You may be able to use the
249Software RAID / MD driver for raid level 5, see the next section for details.
250
251To create the table describing your volume you will need to know each of its
252components and their sizes in sectors, i.e. multiples of 512-byte blocks.
253
254For NT4 fault tolerant volumes you can obtain the sizes using fdisk. So for
255example if one of your partitions is /dev/hda2 you would do:
256
257$ fdisk -ul /dev/hda
258
259Disk /dev/hda: 81.9 GB, 81964302336 bytes
260255 heads, 63 sectors/track, 9964 cylinders, total 160086528 sectors
261Units = sectors of 1 * 512 = 512 bytes
262
263 Device Boot Start End Blocks Id System
264 /dev/hda1 * 63 4209029 2104483+ 83 Linux
265 /dev/hda2 4209030 37768814 16779892+ 86 NTFS
266 /dev/hda3 37768815 46170809 4200997+ 83 Linux
267
268And you would know that /dev/hda2 has a size of 37768814 - 4209030 + 1 =
26933559785 sectors.
270
271For Win2k and later dynamic disks, you can for example use the ldminfo utility
272which is part of the Linux LDM tools (the latest version at the time of
273writing is linux-ldm-0.0.8.tar.bz2). You can download it from:
274 http://www.linux-ntfs.org/
275Simply extract the downloaded archive (tar xvjf linux-ldm-0.0.8.tar.bz2), go
276into it (cd linux-ldm-0.0.8) and change to the test directory (cd test). You
277will find the precompiled (i386) ldminfo utility there. NOTE: You will not be
278able to compile this yourself easily so use the binary version!
279
280Then you would use ldminfo in dump mode to obtain the necessary information:
281
282$ ./ldminfo --dump /dev/hda
283
284This would dump the LDM database found on /dev/hda which describes all of your
285dynamic disks and all the volumes on them. At the bottom you will see the
286VOLUME DEFINITIONS section which is all you really need. You may need to look
287further above to determine which of the disks in the volume definitions is
288which device in Linux. Hint: Run ldminfo on each of your dynamic disks and
289look at the Disk Id close to the top of the output for each (the PRIVATE HEADER
290section). You can then find these Disk Ids in the VBLK DATABASE section in the
291<Disk> components where you will get the LDM Name for the disk that is found in
292the VOLUME DEFINITIONS section.
293
294Note you will also need to enable the LDM driver in the Linux kernel. If your
295distribution did not enable it, you will need to recompile the kernel with it
296enabled. This will create the LDM partitions on each device at boot time. You
297would then use those devices (for /dev/hda they would be /dev/hda1, 2, 3, etc)
298in the Device-Mapper table.
299
300You can also bypass using the LDM driver by using the main device (e.g.
301/dev/hda) and then using the offsets of the LDM partitions into this device as
302the "Start sector of device" when creating the table. Once again ldminfo would
303give you the correct information to do this.
304
305Assuming you know all your devices and their sizes things are easy.
306
307For a linear raid the table would look like this (note all values are in
308512-byte sectors):
309
310--- cut here ---
311# Offset into Size of this Raid type Device Start sector
312# volume device of device
3130 1028161 linear /dev/hda1 0
3141028161 3903762 linear /dev/hdb2 0
3154931923 2103211 linear /dev/hdc1 0
316--- cut here ---
317
318For a striped volume, i.e. raid level 0, you will need to know the chunk size
319you used when creating the volume. Windows uses 64kiB as the default, so it
320will probably be this unless you changes the defaults when creating the array.
321
322For a raid level 0 the table would look like this (note all values are in
323512-byte sectors):
324
325--- cut here ---
326# Offset Size Raid Number Chunk 1st Start 2nd Start
327# into of the type of size Device in Device in
328# volume volume stripes device device
3290 2056320 striped 2 128 /dev/hda1 0 /dev/hdb1 0
330--- cut here ---
331
332If there are more than two devices, just add each of them to the end of the
333line.
334
335Finally, for a mirrored volume, i.e. raid level 1, the table would look like
336this (note all values are in 512-byte sectors):
337
338--- cut here ---
339# Ofs Size Raid Log Number Region Should Number Source Start Target Start
340# in of the type type of log size sync? of Device in Device in
341# vol volume params mirrors Device Device
3420 2056320 mirror core 2 16 nosync 2 /dev/hda1 0 /dev/hdb1 0
343--- cut here ---
344
345If you are mirroring to multiple devices you can specify further targets at the
346end of the line.
347
348Note the "Should sync?" parameter "nosync" means that the two mirrors are
349already in sync which will be the case on a clean shutdown of Windows. If the
350mirrors are not clean, you can specify the "sync" option instead of "nosync"
351and the Device-Mapper driver will then copy the entirety of the "Source Device"
352to the "Target Device" or if you specified multiple target devices to all of
353them.
354
355Once you have your table, save it in a file somewhere (e.g. /etc/ntfsvolume1),
356and hand it over to dmsetup to work with, like so:
357
358$ dmsetup create myvolume1 /etc/ntfsvolume1
359
360You can obviously replace "myvolume1" with whatever name you like.
361
362If it all worked, you will now have the device /dev/device-mapper/myvolume1
363which you can then just use as an argument to the mount command as usual to
364mount the ntfs volume. For example:
365
366$ mount -t ntfs -o ro /dev/device-mapper/myvolume1 /mnt/myvol1
367
368(You need to create the directory /mnt/myvol1 first and of course you can use
369anything you like instead of /mnt/myvol1 as long as it is an existing
370directory.)
371
372It is advisable to do the mount read-only to see if the volume has been setup
373correctly to avoid the possibility of causing damage to the data on the ntfs
374volume.
375
376
377The Software RAID / MD driver
378-----------------------------
379
380An alternative to using the Device-Mapper driver is to use the kernel's
381Software RAID / MD driver. For which you need to set up your /etc/raidtab
382appropriately (see man 5 raidtab).
383
384Linear volume sets, i.e. linear raid, as well as stripe sets, i.e. raid level
3850, have been tested and work fine (though see section "Limitations when using
386the MD driver with NTFS volumes" especially if you want to use linear raid).
387Even though untested, there is no reason why mirrors, i.e. raid level 1, and
388stripes with parity, i.e. raid level 5, should not work, too.
389
390You have to use the "persistent-superblock 0" option for each raid-disk in the
391NTFS volume/stripe you are configuring in /etc/raidtab as the persistent
392superblock used by the MD driver would damage the NTFS volume.
393
394Windows by default uses a stripe chunk size of 64k, so you probably want the
395"chunk-size 64k" option for each raid-disk, too.
396
397For example, if you have a stripe set consisting of two partitions /dev/hda5
398and /dev/hdb1 your /etc/raidtab would look like this:
399
400raiddev /dev/md0
401 raid-level 0
402 nr-raid-disks 2
403 nr-spare-disks 0
404 persistent-superblock 0
405 chunk-size 64k
406 device /dev/hda5
407 raid-disk 0
408 device /dev/hdb1
409 raid-disk 1
410
411For linear raid, just change the raid-level above to "raid-level linear", for
412mirrors, change it to "raid-level 1", and for stripe sets with parity, change
413it to "raid-level 5".
414
415Note for stripe sets with parity you will also need to tell the MD driver
416which parity algorithm to use by specifying the option "parity-algorithm
417which", where you need to replace "which" with the name of the algorithm to
418use (see man 5 raidtab for available algorithms) and you will have to try the
419different available algorithms until you find one that works. Make sure you
420are working read-only when playing with this as you may damage your data
421otherwise. If you find which algorithm works please let us know (email the
422linux-ntfs developers list linux-ntfs-dev@lists.sourceforge.net or drop in on
423IRC in channel #ntfs on the irc.freenode.net network) so we can update this
424documentation.
425
426Once the raidtab is setup, run for example raid0run -a to start all devices or
427raid0run /dev/md0 to start a particular md device, in this case /dev/md0.
428
429Then just use the mount command as usual to mount the ntfs volume using for
430example: mount -t ntfs -o ro /dev/md0 /mnt/myntfsvolume
431
432It is advisable to do the mount read-only to see if the md volume has been
433setup correctly to avoid the possibility of causing damage to the data on the
434ntfs volume.
435
436
437Limitations when using the Software RAID / MD driver
438-----------------------------------------------------
439
440Using the md driver will not work properly if any of your NTFS partitions have
441an odd number of sectors. This is especially important for linear raid as all
442data after the first partition with an odd number of sectors will be offset by
443one or more sectors so if you mount such a partition with write support you
444will cause massive damage to the data on the volume which will only become
445apparent when you try to use the volume again under Windows.
446
447So when using linear raid, make sure that all your partitions have an even
448number of sectors BEFORE attempting to use it. You have been warned!
449
450Even better is to simply use the Device-Mapper for linear raid and then you do
451not have this problem with odd numbers of sectors.