Loading...
1NILFS2
2------
3
4NILFS2 is a log-structured file system (LFS) supporting continuous
5snapshotting. In addition to versioning capability of the entire file
6system, users can even restore files mistakenly overwritten or
7destroyed just a few seconds ago. Since NILFS2 can keep consistency
8like conventional LFS, it achieves quick recovery after system
9crashes.
10
11NILFS2 creates a number of checkpoints every few seconds or per
12synchronous write basis (unless there is no change). Users can select
13significant versions among continuously created checkpoints, and can
14change them into snapshots which will be preserved until they are
15changed back to checkpoints.
16
17There is no limit on the number of snapshots until the volume gets
18full. Each snapshot is mountable as a read-only file system
19concurrently with its writable mount, and this feature is convenient
20for online backup.
21
22The userland tools are included in nilfs-utils package, which is
23available from the following download page. At least "mkfs.nilfs2",
24"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
25cleaner or garbage collector) are required. Details on the tools are
26described in the man pages included in the package.
27
28Project web page: http://www.nilfs.org/en/
29Download page: http://www.nilfs.org/en/download.html
30Git tree web page: http://www.nilfs.org/git/
31List info: http://vger.kernel.org/vger-lists.html#linux-nilfs
32
33Caveats
34=======
35
36Features which NILFS2 does not support yet:
37
38 - atime
39 - extended attributes
40 - POSIX ACLs
41 - quotas
42 - fsck
43 - defragmentation
44
45Mount options
46=============
47
48NILFS2 supports the following mount options:
49(*) == default
50
51barrier(*) This enables/disables the use of write barriers. This
52nobarrier requires an IO stack which can support barriers, and
53 if nilfs gets an error on a barrier write, it will
54 disable again with a warning.
55errors=continue Keep going on a filesystem error.
56errors=remount-ro(*) Remount the filesystem read-only on an error.
57errors=panic Panic and halt the machine if an error occurs.
58cp=n Specify the checkpoint-number of the snapshot to be
59 mounted. Checkpoints and snapshots are listed by lscp
60 user command. Only the checkpoints marked as snapshot
61 are mountable with this option. Snapshot is read-only,
62 so a read-only mount option must be specified together.
63order=relaxed(*) Apply relaxed order semantics that allows modified data
64 blocks to be written to disk without making a
65 checkpoint if no metadata update is going. This mode
66 is equivalent to the ordered data mode of the ext3
67 filesystem except for the updates on data blocks still
68 conserve atomicity. This will improve synchronous
69 write performance for overwriting.
70order=strict Apply strict in-order semantics that preserves sequence
71 of all file operations including overwriting of data
72 blocks. That means, it is guaranteed that no
73 overtaking of events occurs in the recovered file
74 system after a crash.
75norecovery Disable recovery of the filesystem on mount.
76 This disables every write access on the device for
77 read-only mounts or snapshots. This option will fail
78 for r/w mounts on an unclean volume.
79discard This enables/disables the use of discard/TRIM commands.
80nodiscard(*) The discard/TRIM commands are sent to the underlying
81 block device when blocks are freed. This is useful
82 for SSD devices and sparse/thinly-provisioned LUNs.
83
84NILFS2 usage
85============
86
87To use nilfs2 as a local file system, simply:
88
89 # mkfs -t nilfs2 /dev/block_device
90 # mount -t nilfs2 /dev/block_device /dir
91
92This will also invoke the cleaner through the mount helper program
93(mount.nilfs2).
94
95Checkpoints and snapshots are managed by the following commands.
96Their manpages are included in the nilfs-utils package above.
97
98 lscp list checkpoints or snapshots.
99 mkcp make a checkpoint or a snapshot.
100 chcp change an existing checkpoint to a snapshot or vice versa.
101 rmcp invalidate specified checkpoint(s).
102
103To mount a snapshot,
104
105 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir
106
107where <cno> is the checkpoint number of the snapshot.
108
109To unmount the NILFS2 mount point or snapshot, simply:
110
111 # umount /dir
112
113Then, the cleaner daemon is automatically shut down by the umount
114helper program (umount.nilfs2).
115
116Disk format
117===========
118
119A nilfs2 volume is equally divided into a number of segments except
120for the super block (SB) and segment #0. A segment is the container
121of logs. Each log is composed of summary information blocks, payload
122blocks, and an optional super root block (SR):
123
124 ______________________________________________________
125 | |SB| | Segment | Segment | Segment | ... | Segment | |
126 |_|__|_|____0____|____1____|____2____|_____|____N____|_|
127 0 +1K +4K +8M +16M +24M +(8MB x N)
128 . . (Typical offsets for 4KB-block)
129 . .
130 .______________________.
131 | log | log |... | log |
132 |__1__|__2__|____|__m__|
133 . .
134 . .
135 . .
136 .______________________________.
137 | Summary | Payload blocks |SR|
138 |_blocks__|_________________|__|
139
140The payload blocks are organized per file, and each file consists of
141data blocks and B-tree node blocks:
142
143 |<--- File-A --->|<--- File-B --->|
144 _______________________________________________________________
145 | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
146 _|_____________|_______________|_____________|_______________|_
147
148
149Since only the modified blocks are written in the log, it may have
150files without data blocks or B-tree node blocks.
151
152The organization of the blocks is recorded in the summary information
153blocks, which contains a header structure (nilfs_segment_summary), per
154file structures (nilfs_finfo), and per block structures (nilfs_binfo):
155
156 _________________________________________________________________________
157 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
158 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___
159
160
161The logs include regular files, directory files, symbolic link files
162and several meta data files. The mata data files are the files used
163to maintain file system meta data. The current version of NILFS2 uses
164the following meta data files:
165
166 1) Inode file (ifile) -- Stores on-disk inodes
167 2) Checkpoint file (cpfile) -- Stores checkpoints
168 3) Segment usage file (sufile) -- Stores allocation state of segments
169 4) Data address translation file -- Maps virtual block numbers to usual
170 (DAT) block numbers. This file serves to
171 make on-disk blocks relocatable.
172
173The following figure shows a typical organization of the logs:
174
175 _________________________________________________________________________
176 | Summary | regular file | file | ... | ifile | cpfile | sufile | DAT |SR|
177 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|
178
179
180To stride over segment boundaries, this sequence of files may be split
181into multiple logs. The sequence of logs that should be treated as
182logically one log, is delimited with flags marked in the segment
183summary. The recovery code of nilfs2 looks this boundary information
184to ensure atomicity of updates.
185
186The super root block is inserted for every checkpoints. It includes
187three special inodes, inodes for the DAT, cpfile, and sufile. Inodes
188of regular files, directories, symlinks and other special files, are
189included in the ifile. The inode of ifile itself is included in the
190corresponding checkpoint entry in the cpfile. Thus, the hierarchy
191among NILFS2 files can be depicted as follows:
192
193 Super block (SB)
194 |
195 v
196 Super root block (the latest cno=xx)
197 |-- DAT
198 |-- sufile
199 `-- cpfile
200 |-- ifile (cno=c1)
201 |-- ifile (cno=c2) ---- file (ino=i1)
202 : : |-- file (ino=i2)
203 `-- ifile (cno=xx) |-- file (ino=i3)
204 : :
205 `-- file (ino=yy)
206 ( regular file, directory, or symlink )
207
208For detail on the format of each file, please see include/linux/nilfs2_fs.h.
1NILFS2
2------
3
4NILFS2 is a log-structured file system (LFS) supporting continuous
5snapshotting. In addition to versioning capability of the entire file
6system, users can even restore files mistakenly overwritten or
7destroyed just a few seconds ago. Since NILFS2 can keep consistency
8like conventional LFS, it achieves quick recovery after system
9crashes.
10
11NILFS2 creates a number of checkpoints every few seconds or per
12synchronous write basis (unless there is no change). Users can select
13significant versions among continuously created checkpoints, and can
14change them into snapshots which will be preserved until they are
15changed back to checkpoints.
16
17There is no limit on the number of snapshots until the volume gets
18full. Each snapshot is mountable as a read-only file system
19concurrently with its writable mount, and this feature is convenient
20for online backup.
21
22The userland tools are included in nilfs-utils package, which is
23available from the following download page. At least "mkfs.nilfs2",
24"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
25cleaner or garbage collector) are required. Details on the tools are
26described in the man pages included in the package.
27
28Project web page: http://nilfs.sourceforge.net/
29Download page: http://nilfs.sourceforge.net/en/download.html
30List info: http://vger.kernel.org/vger-lists.html#linux-nilfs
31
32Caveats
33=======
34
35Features which NILFS2 does not support yet:
36
37 - atime
38 - extended attributes
39 - POSIX ACLs
40 - quotas
41 - fsck
42 - defragmentation
43
44Mount options
45=============
46
47NILFS2 supports the following mount options:
48(*) == default
49
50barrier(*) This enables/disables the use of write barriers. This
51nobarrier requires an IO stack which can support barriers, and
52 if nilfs gets an error on a barrier write, it will
53 disable again with a warning.
54errors=continue Keep going on a filesystem error.
55errors=remount-ro(*) Remount the filesystem read-only on an error.
56errors=panic Panic and halt the machine if an error occurs.
57cp=n Specify the checkpoint-number of the snapshot to be
58 mounted. Checkpoints and snapshots are listed by lscp
59 user command. Only the checkpoints marked as snapshot
60 are mountable with this option. Snapshot is read-only,
61 so a read-only mount option must be specified together.
62order=relaxed(*) Apply relaxed order semantics that allows modified data
63 blocks to be written to disk without making a
64 checkpoint if no metadata update is going. This mode
65 is equivalent to the ordered data mode of the ext3
66 filesystem except for the updates on data blocks still
67 conserve atomicity. This will improve synchronous
68 write performance for overwriting.
69order=strict Apply strict in-order semantics that preserves sequence
70 of all file operations including overwriting of data
71 blocks. That means, it is guaranteed that no
72 overtaking of events occurs in the recovered file
73 system after a crash.
74norecovery Disable recovery of the filesystem on mount.
75 This disables every write access on the device for
76 read-only mounts or snapshots. This option will fail
77 for r/w mounts on an unclean volume.
78discard This enables/disables the use of discard/TRIM commands.
79nodiscard(*) The discard/TRIM commands are sent to the underlying
80 block device when blocks are freed. This is useful
81 for SSD devices and sparse/thinly-provisioned LUNs.
82
83Ioctls
84======
85
86There is some NILFS2 specific functionality which can be accessed by applications
87through the system call interfaces. The list of all NILFS2 specific ioctls are
88shown in the table below.
89
90Table of NILFS2 specific ioctls
91..............................................................................
92 Ioctl Description
93 NILFS_IOCTL_CHANGE_CPMODE Change mode of given checkpoint between
94 checkpoint and snapshot state. This ioctl is
95 used in chcp and mkcp utilities.
96
97 NILFS_IOCTL_DELETE_CHECKPOINT Remove checkpoint from NILFS2 file system.
98 This ioctl is used in rmcp utility.
99
100 NILFS_IOCTL_GET_CPINFO Return info about requested checkpoints. This
101 ioctl is used in lscp utility and by
102 nilfs_cleanerd daemon.
103
104 NILFS_IOCTL_GET_CPSTAT Return checkpoints statistics. This ioctl is
105 used by lscp, rmcp utilities and by
106 nilfs_cleanerd daemon.
107
108 NILFS_IOCTL_GET_SUINFO Return segment usage info about requested
109 segments. This ioctl is used in lssu,
110 nilfs_resize utilities and by nilfs_cleanerd
111 daemon.
112
113 NILFS_IOCTL_SET_SUINFO Modify segment usage info of requested
114 segments. This ioctl is used by
115 nilfs_cleanerd daemon to skip unnecessary
116 cleaning operation of segments and reduce
117 performance penalty or wear of flash device
118 due to redundant move of in-use blocks.
119
120 NILFS_IOCTL_GET_SUSTAT Return segment usage statistics. This ioctl
121 is used in lssu, nilfs_resize utilities and
122 by nilfs_cleanerd daemon.
123
124 NILFS_IOCTL_GET_VINFO Return information on virtual block addresses.
125 This ioctl is used by nilfs_cleanerd daemon.
126
127 NILFS_IOCTL_GET_BDESCS Return information about descriptors of disk
128 block numbers. This ioctl is used by
129 nilfs_cleanerd daemon.
130
131 NILFS_IOCTL_CLEAN_SEGMENTS Do garbage collection operation in the
132 environment of requested parameters from
133 userspace. This ioctl is used by
134 nilfs_cleanerd daemon.
135
136 NILFS_IOCTL_SYNC Make a checkpoint. This ioctl is used in
137 mkcp utility.
138
139 NILFS_IOCTL_RESIZE Resize NILFS2 volume. This ioctl is used
140 by nilfs_resize utility.
141
142 NILFS_IOCTL_SET_ALLOC_RANGE Define lower limit of segments in bytes and
143 upper limit of segments in bytes. This ioctl
144 is used by nilfs_resize utility.
145
146NILFS2 usage
147============
148
149To use nilfs2 as a local file system, simply:
150
151 # mkfs -t nilfs2 /dev/block_device
152 # mount -t nilfs2 /dev/block_device /dir
153
154This will also invoke the cleaner through the mount helper program
155(mount.nilfs2).
156
157Checkpoints and snapshots are managed by the following commands.
158Their manpages are included in the nilfs-utils package above.
159
160 lscp list checkpoints or snapshots.
161 mkcp make a checkpoint or a snapshot.
162 chcp change an existing checkpoint to a snapshot or vice versa.
163 rmcp invalidate specified checkpoint(s).
164
165To mount a snapshot,
166
167 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir
168
169where <cno> is the checkpoint number of the snapshot.
170
171To unmount the NILFS2 mount point or snapshot, simply:
172
173 # umount /dir
174
175Then, the cleaner daemon is automatically shut down by the umount
176helper program (umount.nilfs2).
177
178Disk format
179===========
180
181A nilfs2 volume is equally divided into a number of segments except
182for the super block (SB) and segment #0. A segment is the container
183of logs. Each log is composed of summary information blocks, payload
184blocks, and an optional super root block (SR):
185
186 ______________________________________________________
187 | |SB| | Segment | Segment | Segment | ... | Segment | |
188 |_|__|_|____0____|____1____|____2____|_____|____N____|_|
189 0 +1K +4K +8M +16M +24M +(8MB x N)
190 . . (Typical offsets for 4KB-block)
191 . .
192 .______________________.
193 | log | log |... | log |
194 |__1__|__2__|____|__m__|
195 . .
196 . .
197 . .
198 .______________________________.
199 | Summary | Payload blocks |SR|
200 |_blocks__|_________________|__|
201
202The payload blocks are organized per file, and each file consists of
203data blocks and B-tree node blocks:
204
205 |<--- File-A --->|<--- File-B --->|
206 _______________________________________________________________
207 | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
208 _|_____________|_______________|_____________|_______________|_
209
210
211Since only the modified blocks are written in the log, it may have
212files without data blocks or B-tree node blocks.
213
214The organization of the blocks is recorded in the summary information
215blocks, which contains a header structure (nilfs_segment_summary), per
216file structures (nilfs_finfo), and per block structures (nilfs_binfo):
217
218 _________________________________________________________________________
219 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
220 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___
221
222
223The logs include regular files, directory files, symbolic link files
224and several meta data files. The mata data files are the files used
225to maintain file system meta data. The current version of NILFS2 uses
226the following meta data files:
227
228 1) Inode file (ifile) -- Stores on-disk inodes
229 2) Checkpoint file (cpfile) -- Stores checkpoints
230 3) Segment usage file (sufile) -- Stores allocation state of segments
231 4) Data address translation file -- Maps virtual block numbers to usual
232 (DAT) block numbers. This file serves to
233 make on-disk blocks relocatable.
234
235The following figure shows a typical organization of the logs:
236
237 _________________________________________________________________________
238 | Summary | regular file | file | ... | ifile | cpfile | sufile | DAT |SR|
239 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|
240
241
242To stride over segment boundaries, this sequence of files may be split
243into multiple logs. The sequence of logs that should be treated as
244logically one log, is delimited with flags marked in the segment
245summary. The recovery code of nilfs2 looks this boundary information
246to ensure atomicity of updates.
247
248The super root block is inserted for every checkpoints. It includes
249three special inodes, inodes for the DAT, cpfile, and sufile. Inodes
250of regular files, directories, symlinks and other special files, are
251included in the ifile. The inode of ifile itself is included in the
252corresponding checkpoint entry in the cpfile. Thus, the hierarchy
253among NILFS2 files can be depicted as follows:
254
255 Super block (SB)
256 |
257 v
258 Super root block (the latest cno=xx)
259 |-- DAT
260 |-- sufile
261 `-- cpfile
262 |-- ifile (cno=c1)
263 |-- ifile (cno=c2) ---- file (ino=i1)
264 : : |-- file (ino=i2)
265 `-- ifile (cno=xx) |-- file (ino=i3)
266 : :
267 `-- file (ino=yy)
268 ( regular file, directory, or symlink )
269
270For detail on the format of each file, please see include/linux/nilfs2_fs.h.