[RFC v2 43/83] Log operation: in-place update log entry

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> To in-place update a log entry, NOVA starts a lite transaction to journal the log entry, then performs update and commits the transaction. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/inode.h | 12 fs/nova/l

[RFC v2 43/83] Log operation: in-place update log entry

2018-03-10 Thread Andiry Xu
From: Andiry Xu To in-place update a log entry, NOVA starts a lite transaction to journal the log entry, then performs update and commits the transaction. Signed-off-by: Andiry Xu --- fs/nova/inode.h | 12 fs/nova/log.c | 183

[RFC v2 35/83] Journal: Lite journal helper routines.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/Makefile | 2 +- fs/nova/journal.c | 108 ++ 2 files changed, 109 insertions(+), 1 deletion(-) create mode 100644 fs/nova/journal.c

[RFC v2 35/83] Journal: Lite journal helper routines.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/journal.c | 108 ++ 2 files changed, 109 insertions(+), 1 deletion(-) create mode 100644 fs/nova/journal.c diff --git a/fs/nova/Makefile b/fs/nova/Makefile

[RFC v2 37/83] Journal: Lite journal create and commit.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA uses lite journal to perform light weight transaction. Instead of journaling metadata/data changes directly, NOVA first append updates to each inode's log, and then journal the log tail pointers to make sure all the logs are updated atomically. For

[RFC v2 40/83] Log operation: file write entry append.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA performs writes by appending file write entries to the log. A file write entry is the metadata of a write operation, and contains pointers to the data blocks. A single write operation may append multiple file write entries to the log, if the all

[RFC v2 37/83] Journal: Lite journal create and commit.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA uses lite journal to perform light weight transaction. Instead of journaling metadata/data changes directly, NOVA first append updates to each inode's log, and then journal the log tail pointers to make sure all the logs are updated atomically. For inode creation

[RFC v2 40/83] Log operation: file write entry append.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA performs writes by appending file write entries to the log. A file write entry is the metadata of a write operation, and contains pointers to the data blocks. A single write operation may append multiple file write entries to the log, if the allocator cannot provide enough

[RFC v2 41/83] Log operation: setattr entry append

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA appends a setattr entry to the log upon inode modification operations: set size, chmod, etc. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/log.c | 64 +++ 1 file changed, 6

[RFC v2 41/83] Log operation: setattr entry append

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA appends a setattr entry to the log upon inode modification operations: set size, chmod, etc. Signed-off-by: Andiry Xu --- fs/nova/log.c | 64 +++ 1 file changed, 64 insertions(+) diff --git a/fs/nova/log.c b/fs/nova

[RFC v2 44/83] Log operation: invalidate log entries

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> After new log entries are appended to the log, old log entries can be marked invalid to faciliate garbage collection. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/log.c | 160 ++

[RFC v2 44/83] Log operation: invalidate log entries

2018-03-10 Thread Andiry Xu
From: Andiry Xu After new log entries are appended to the log, old log entries can be marked invalid to faciliate garbage collection. Signed-off-by: Andiry Xu --- fs/nova/log.c | 160 + fs/nova/log.h | 4 ++ fs/nova/nova.h | 12

[RFC v2 42/83] Log operation: link change append.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA appends link change entries to atomically update link count and ctime. This occurs in link, unlink and rmdir. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/log.c | 52 fs/nov

[RFC v2 42/83] Log operation: link change append.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA appends link change entries to atomically update link count and ctime. This occurs in link, unlink and rmdir. Signed-off-by: Andiry Xu --- fs/nova/log.c | 52 fs/nova/log.h | 3 +++ 2 files changed, 55 insertions

[RFC v2 47/83] Dir: Add initial dentries when initializing a directory inode log.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> For root directory and newly created directory via mkdir(), we append . and .. dentries to the directory inode log. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/dir.c | 82 ++

[RFC v2 47/83] Dir: Add initial dentries when initializing a directory inode log.

2018-03-10 Thread Andiry Xu
From: Andiry Xu For root directory and newly created directory via mkdir(), we append . and .. dentries to the directory inode log. Signed-off-by: Andiry Xu --- fs/nova/dir.c | 82 + fs/nova/nova.h | 2 ++ fs/nova/super.c | 5

[RFC v2 46/83] Dir: Add Directory radix tree insert/remove methods.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA uses Hash to quickly locate dentry in the directory inode log. The key is the hash of the filename, the value is the dentry. Currently hash collision is ignored, and the radix tree may occupy large memory space with huge directories. Consi

[RFC v2 46/83] Dir: Add Directory radix tree insert/remove methods.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA uses Hash to quickly locate dentry in the directory inode log. The key is the hash of the filename, the value is the dentry. Currently hash collision is ignored, and the radix tree may occupy large memory space with huge directories. Considering replacing it in the future

[RFC v2 50/83] Inode: Add nova_evict_inode.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> If the inode still have links, release the DRAM resource (radix tree, etc). Otherwise reclaim data pages and log pages. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/inode.c | 257 +

[RFC v2 50/83] Inode: Add nova_evict_inode.

2018-03-10 Thread Andiry Xu
From: Andiry Xu If the inode still have links, release the DRAM resource (radix tree, etc). Otherwise reclaim data pages and log pages. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 257 +++- fs/nova/inode.h | 5 ++ fs/nova/log.h | 7

[RFC v2 45/83] Log operation: file inode log lookup and assign

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> After NOVA appends file write entry to commit new writes, it updates the file offset radix tree, finds the old entries (if overwrite) and reclaims the stale data blocks. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/

[RFC v2 45/83] Log operation: file inode log lookup and assign

2018-03-10 Thread Andiry Xu
From: Andiry Xu After NOVA appends file write entry to commit new writes, it updates the file offset radix tree, finds the old entries (if overwrite) and reclaims the stale data blocks. Signed-off-by: Andiry Xu --- fs/nova/log.c | 108

[RFC v2 49/83] Dir: Append create/remove dentry.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA adds or removes a directory/file by appending a dentry to the parent directory's log. Dentry contains filename and inode number. A positive inode number indicates a create(valid) dentry, and a dentry with inode number zero is a remove dentry. NO

[RFC v2 51/83] Rebuild: directory inode.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> When vfs issues a read inode command, or when the inode is newly allocated, walk through the inode log to rebuild inode information and the radix tree. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/inode.h | 15 +++ fs/nova/nov

[RFC v2 49/83] Dir: Append create/remove dentry.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA adds or removes a directory/file by appending a dentry to the parent directory's log. Dentry contains filename and inode number. A positive inode number indicates a create(valid) dentry, and a dentry with inode number zero is a remove dentry. NOVA can also inplace update

[RFC v2 51/83] Rebuild: directory inode.

2018-03-10 Thread Andiry Xu
From: Andiry Xu When vfs issues a read inode command, or when the inode is newly allocated, walk through the inode log to rebuild inode information and the radix tree. Signed-off-by: Andiry Xu --- fs/nova/inode.h | 15 +++ fs/nova/nova.h| 21 fs/nova/rebuild.c | 329

[RFC v2 52/83] Rebuild: file inode.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Rebuild file inode metadata and radix tree on read_inode. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/log.h | 4 ++ fs/nova/rebuild.c | 124 ++ 2 files changed, 128 inserti

[RFC v2 52/83] Rebuild: file inode.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Rebuild file inode metadata and radix tree on read_inode. Signed-off-by: Andiry Xu --- fs/nova/log.h | 4 ++ fs/nova/rebuild.c | 124 ++ 2 files changed, 128 insertions(+) diff --git a/fs/nova/log.h b/fs/nova/log.h

[RFC v2 55/83] Namei: mkdir

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA mkdir is similar to create. The difference is NOVA will allocate log page for the newly created directory, and append init dentries. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/

[RFC v2 58/83] Namei: rename

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Rename is the most cpmplex namei operation. The target dir may be different from the source dir, and the target inode may exist. Rename involves up to four inodes, and NOVA uses rename transation to atomically update all the affected inodes. Sign

[RFC v2 55/83] Namei: mkdir

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA mkdir is similar to create. The difference is NOVA will allocate log page for the newly created directory, and append init dentries. Signed-off-by: Andiry Xu --- fs/nova/namei.c | 74 + 1 file changed, 74 insertions

[RFC v2 58/83] Namei: rename

2018-03-10 Thread Andiry Xu
From: Andiry Xu Rename is the most cpmplex namei operation. The target dir may be different from the source dir, and the target inode may exist. Rename involves up to four inodes, and NOVA uses rename transation to atomically update all the affected inodes. Signed-off-by: Andiry Xu --- fs

[RFC v2 57/83] Namei: rmdir

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Similar to unlink. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/namei.c | 105 1 file changed, 105 insertions(+) diff --git a/fs/nova/namei.c b/fs/nova/namei.c index 360d

[RFC v2 57/83] Namei: rmdir

2018-03-10 Thread Andiry Xu
From: Andiry Xu Similar to unlink. Signed-off-by: Andiry Xu --- fs/nova/namei.c | 105 1 file changed, 105 insertions(+) diff --git a/fs/nova/namei.c b/fs/nova/namei.c index 360d716..4bf6396 100644 --- a/fs/nova/namei.c +++ b/fs/nova

[RFC v2 56/83] Namei: link and unlink.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> For link change operations, NOVA appends a link change entry to the affected inode's log, and uses lite transaction to atomically commit changes to multiple logs. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/n

[RFC v2 56/83] Namei: link and unlink.

2018-03-10 Thread Andiry Xu
From: Andiry Xu For link change operations, NOVA appends a link change entry to the affected inode's log, and uses lite transaction to atomically commit changes to multiple logs. Signed-off-by: Andiry Xu --- fs/nova/namei.c | 159 1

[RFC v2 59/83] Namei: setattr

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Add notify_change for setattr operations. Truncate the file blocks if the file is shrunk. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/inode.c | 180 fs/nova/inode.h |

[RFC v2 60/83] Add special inode operations.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/inode.c | 2 ++ fs/nova/namei.c | 5 + fs/nova/nova.h | 1 + 3 files changed, 8 insertions(+) diff --git a/fs/nova/inode.c b/fs/nova/inode.c index 2092a55..0e9ab4b 100644 --- a/fs/

[RFC v2 60/83] Add special inode operations.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/inode.c | 2 ++ fs/nova/namei.c | 5 + fs/nova/nova.h | 1 + 3 files changed, 8 insertions(+) diff --git a/fs/nova/inode.c b/fs/nova/inode.c index 2092a55..0e9ab4b 100644 --- a/fs/nova/inode.c +++ b/fs/nova/inode.c @@ -239,6 +239,7

[RFC v2 59/83] Namei: setattr

2018-03-10 Thread Andiry Xu
From: Andiry Xu Add notify_change for setattr operations. Truncate the file blocks if the file is shrunk. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 180 fs/nova/inode.h | 1 + fs/nova/namei.c | 2 + 3 files changed, 183

[RFC v2 61/83] Super: Add nova_export_ops.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/super.c | 48 1 file changed, 48 insertions(+) diff --git a/fs/nova/super.c b/fs/nova/super.c index daf3270..0847e57 100644 --- a/fs/nova/s

[RFC v2 54/83] Namei: create and mknod.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA allocates and initializes a new inode, and appends a dentry to the directory's log. Then NOVA creates a transaction to commit both changes atomically: update the directory log tail pointer and validate the new inode. Signed-off-by: Andiry X

[RFC v2 62/83] File: getattr and file inode operations

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/Makefile | 2 +- fs/nova/file.c | 31 +++ fs/nova/inode.c | 25 + fs/nova/inode.h | 2 ++ fs/nova/nova.h | 3 +++ 5 files

[RFC v2 54/83] Namei: create and mknod.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA allocates and initializes a new inode, and appends a dentry to the directory's log. Then NOVA creates a transaction to commit both changes atomically: update the directory log tail pointer and validate the new inode. Signed-off-by: Andiry Xu --- fs/nova/namei.c | 141

[RFC v2 62/83] File: getattr and file inode operations

2018-03-10 Thread Andiry Xu
From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/file.c | 31 +++ fs/nova/inode.c | 25 + fs/nova/inode.h | 2 ++ fs/nova/nova.h | 3 +++ 5 files changed, 62 insertions(+), 1 deletion(-) create mode

[RFC v2 61/83] Super: Add nova_export_ops.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/super.c | 48 1 file changed, 48 insertions(+) diff --git a/fs/nova/super.c b/fs/nova/super.c index daf3270..0847e57 100644 --- a/fs/nova/super.c +++ b/fs/nova/super.c @@ -51,6 +51,7

[RFC v2 53/83] Namei: lookup.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA lookup the inode number by searching the radix tree with the filename hash value and locating the corresponding dentry on the log. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/Makefile | 3 +- fs/nova/inode.c | 2 ++ fs/

[RFC v2 63/83] File operation: llseek.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Search the file radix tree to find hold or data. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/file.c | 47 +++ fs/nova/inode.c | 113 fs/nova/inode.h |

[RFC v2 53/83] Namei: lookup.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA lookup the inode number by searching the radix tree with the filename hash value and locating the corresponding dentry on the log. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 3 +- fs/nova/inode.c | 2 ++ fs/nova/namei.c | 97

[RFC v2 63/83] File operation: llseek.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Search the file radix tree to find hold or data. Signed-off-by: Andiry Xu --- fs/nova/file.c | 47 +++ fs/nova/inode.c | 113 fs/nova/inode.h | 1 + fs/nova/nova.h | 1 + 4 files changed, 162

[RFC v2 66/83] Super: Add file write item cache.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> nova_file_write_item combines a file write item with a list head. NOVA uses a linked list of file write items to describe a write operation. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/

[RFC v2 66/83] Super: Add file write item cache.

2018-03-10 Thread Andiry Xu
From: Andiry Xu nova_file_write_item combines a file write item with a list head. NOVA uses a linked list of file write items to describe a write operation. Signed-off-by: Andiry Xu --- fs/nova/super.c | 43 ++- fs/nova/super.h | 3 +++ 2 files changed

[RFC v2 64/83] File operation: open, fsync, flush.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA persists file metadata and data before returning to the user space. Hence, fsync is a no-op if the file is not mmaped. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/file.c | 50 ++

[RFC v2 64/83] File operation: open, fsync, flush.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA persists file metadata and data before returning to the user space. Hence, fsync is a no-op if the file is not mmaped. Signed-off-by: Andiry Xu --- fs/nova/file.c | 50 ++ 1 file changed, 50 insertions(+) diff --git a/fs

[RFC v2 67/83] Dax: commit list of file write items to log.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Given a list of file write items, NOVA commits them by appending each file write entry to the log, and then updates the radix tree to point to these new entries, and updates log tail pointer to commit all the writes atomically. If the items are all

[RFC v2 67/83] Dax: commit list of file write items to log.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Given a list of file write items, NOVA commits them by appending each file write entry to the log, and then updates the radix tree to point to these new entries, and updates log tail pointer to commit all the writes atomically. If the items are allocated on heap, free them

[RFC v2 65/83] File operation: read.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA is a DAX file system and does not use page cache. For read, NOVA looks up the file write entry by searching the radix tree, and copies data from pmem pages to user buffer directly. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/

[RFC v2 65/83] File operation: read.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA is a DAX file system and does not use page cache. For read, NOVA looks up the file write entry by searching the radix tree, and copies data from pmem pages to user buffer directly. Signed-off-by: Andiry Xu --- fs/nova/file.c | 144

[RFC v2 68/83] File operation: copy-on-write write.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> If the file is not mmaped, NOVA performs copy-on-write. The CoW is composed of parts: 1. Allocate contiguous data pages. 2. Copy data from user buffer to the data pages. If the write is not aligned to page size, also copy data from existing pmem

[RFC v2 68/83] File operation: copy-on-write write.

2018-03-10 Thread Andiry Xu
From: Andiry Xu If the file is not mmaped, NOVA performs copy-on-write. The CoW is composed of parts: 1. Allocate contiguous data pages. 2. Copy data from user buffer to the data pages. If the write is not aligned to page size, also copy data from existing pmem pages. 3. Allocate

[RFC v2 70/83] File operation: Inplace write.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> If the user specifies inplace updates, or the file is mmaped, NOVA performs inplace writes. The trick is dax page fault can occur concurrently with inplace writes, and allocate new blocks. Also, inplace write memcpy may trigger page fault (xfstes

[RFC v2 70/83] File operation: Inplace write.

2018-03-10 Thread Andiry Xu
From: Andiry Xu If the user specifies inplace updates, or the file is mmaped, NOVA performs inplace writes. The trick is dax page fault can occur concurrently with inplace writes, and allocate new blocks. Also, inplace write memcpy may trigger page fault (xfstests 248). Since page fault may

[RFC v2 69/83] Super: Add module param inplace_data_updates.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Provide inplace data updates option if people prefer inplace updates to copy-on-write. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/nova.h | 1 + fs/nova/super.c | 7 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --g

[RFC v2 69/83] Super: Add module param inplace_data_updates.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Provide inplace data updates option if people prefer inplace updates to copy-on-write. Signed-off-by: Andiry Xu --- fs/nova/nova.h | 1 + fs/nova/super.c | 7 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/nova/nova.h b/fs/nova/nova.h index 1c2205e

[RFC v2 73/83] Dax: Add iomap operations.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> The key of iomap is dax_get_blocks(). It first takes the read lock and lookup the block; if the block is missing, it takes write lock, check again and allocate the new block if needed. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/

[RFC v2 71/83] Symlink support.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA alloates two blocks for symlink inode: One for inode log, and the other one is a data block, storing symname. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/Makefile | 2 +- fs/nova/inode.c | 2 + fs/nova/nam

[RFC v2 71/83] Symlink support.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA alloates two blocks for symlink inode: One for inode log, and the other one is a data block, storing symname. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/inode.c | 2 + fs/nova/namei.c | 70 fs/nova/nova.h| 5

[RFC v2 73/83] Dax: Add iomap operations.

2018-03-10 Thread Andiry Xu
From: Andiry Xu The key of iomap is dax_get_blocks(). It first takes the read lock and lookup the block; if the block is missing, it takes write lock, check again and allocate the new block if needed. Signed-off-by: Andiry Xu --- fs/nova/dax.c | 184

[RFC v2 72/83] File operation: fallocate.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Fallocate works similar as writes, allocating zeroed blocked for the holes in the request region. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/file.c | 148 + fs/nova/nova.h

[RFC v2 75/83] File operation: read/write iter.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> They use the iomap framework to do read/write. Due to software overheads they are slower than dax read/write. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/file.c | 65 ++ 1 file

[RFC v2 72/83] File operation: fallocate.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Fallocate works similar as writes, allocating zeroed blocked for the holes in the request region. Signed-off-by: Andiry Xu --- fs/nova/file.c | 148 + fs/nova/nova.h | 5 ++ 2 files changed, 153 insertions(+) diff

[RFC v2 75/83] File operation: read/write iter.

2018-03-10 Thread Andiry Xu
From: Andiry Xu They use the iomap framework to do read/write. Due to software overheads they are slower than dax read/write. Signed-off-by: Andiry Xu --- fs/nova/file.c | 65 ++ 1 file changed, 65 insertions(+) diff --git a/fs/nova

[RFC v2 74/83] File operation: Mmap.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA uses the iomap framework to support mmap operation. Currently it does not support huge page mmap. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/dax.c | 53 + fs/nova

[RFC v2 74/83] File operation: Mmap.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA uses the iomap framework to support mmap operation. Currently it does not support huge page mmap. Signed-off-by: Andiry Xu --- fs/nova/dax.c | 53 + fs/nova/file.c | 25 + fs/nova/nova.h | 1

[RFC v2 76/83] Ioctl support.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA appends link change entry to the inode log to implement SETFLAGS and SETVERSION. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/Makefile | 4 +- fs/nova/dir.c| 4 ++ fs/nova/file.c | 4 ++ fs/nova/inode.h | 2 + fs/

[RFC v2 76/83] Ioctl support.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA appends link change entry to the inode log to implement SETFLAGS and SETVERSION. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 4 +- fs/nova/dir.c| 4 ++ fs/nova/file.c | 4 ++ fs/nova/inode.h | 2 + fs/nova/ioctl.c | 184

[RFC v2 80/83] Failure recovery: bitmap operations.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Upon system failure, NOVA needs to scan all the inode logs to rebuild the allocator. During the scanning, NOVA stores allocated log/data pages in a bitmap, and uses the bitmap to rebuild the allocator once scan finishes. Signed-off-by: Andiry X

[RFC v2 80/83] Failure recovery: bitmap operations.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Upon system failure, NOVA needs to scan all the inode logs to rebuild the allocator. During the scanning, NOVA stores allocated log/data pages in a bitmap, and uses the bitmap to rebuild the allocator once scan finishes. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 252

[RFC v2 79/83] Normal recovery.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Upon umount, NOVA stores the allocator information and the inuse inode list in reserved inodes. During remount, NOVA reads these information and rebuild the allocator and inuse inode list DRAM data structures. Signed-off-by: Andiry Xu <jix...@cs

[RFC v2 79/83] Normal recovery.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Upon umount, NOVA stores the allocator information and the inuse inode list in reserved inodes. During remount, NOVA reads these information and rebuild the allocator and inuse inode list DRAM data structures. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 266

[RFC v2 83/83] Sysfs support.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Sysfs support allows user to get/post information of running NOVA instance. After mount, NOVA creates four entries under proc directory /proc/fs/nova/pmem#/: timing_statsIO_statsallocator gc Show NOVA file operation timing statistic

[RFC v2 81/83] Failure recovery: Inode pages recovery routines.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> For each inode, NOVA traverses the inode log and records the pages allocated in the bitmap. For directory inode, NOVA only set the log pages. For file and symlink inodes, NOVA needs to set the data pages. NOVA divides the file into 1GB zones, and r

[RFC v2 83/83] Sysfs support.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Sysfs support allows user to get/post information of running NOVA instance. After mount, NOVA creates four entries under proc directory /proc/fs/nova/pmem#/: timing_statsIO_statsallocator gc Show NOVA file operation timing statistics: cat /proc/fs/NOVA/pmem

[RFC v2 81/83] Failure recovery: Inode pages recovery routines.

2018-03-10 Thread Andiry Xu
From: Andiry Xu For each inode, NOVA traverses the inode log and records the pages allocated in the bitmap. For directory inode, NOVA only set the log pages. For file and symlink inodes, NOVA needs to set the data pages. NOVA divides the file into 1GB zones, and records the pages fall

[RFC v2 82/83] Failure recovery: Per-CPU recovery.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA starts a recovery thread on each CPU, and scans all the inodes in a parallel way. It recovers the inode inuse list during the scan as well. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/bb

[RFC v2 82/83] Failure recovery: Per-CPU recovery.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA starts a recovery thread on each CPU, and scans all the inodes in a parallel way. It recovers the inode inuse list during the scan as well. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 396 +++ 1 file changed, 396

[RFC v2 77/83] GC: Fast garbage collection.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA cleans and compacts the log when the log is full. The log is a linked list of 4KB pmem pages, and NOVA performs fast garbage collection by deleting dead log pages (all the entries are invalid) from the linked list. Example: I = Invalid, V = Valid

[RFC v2 77/83] GC: Fast garbage collection.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA cleans and compacts the log when the log is full. The log is a linked list of 4KB pmem pages, and NOVA performs fast garbage collection by deleting dead log pages (all the entries are invalid) from the linked list. Example: I = Invalid, V = Valid VIIV -> ->

[RFC v2 78/83] GC: Thorough garbage collection.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> After fast gc, if the valid log entries still account for less than the half of the log size, NOVA starts thorough garbage collection, allocates a new log, copies the live log entries to it, and switches to the new log atomically. The radix tree

[RFC v2 78/83] GC: Thorough garbage collection.

2018-03-10 Thread Andiry Xu
From: Andiry Xu After fast gc, if the valid log entries still account for less than the half of the log size, NOVA starts thorough garbage collection, allocates a new log, copies the live log entries to it, and switches to the new log atomically. The radix tree needs to be updated to point

[RFC v2 48/83] Dir: Readdir operation.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA reads the directory by traversing the log and reports the valid dentries. Valid dentris have inode number greater than zero, meaning it's a create dentry. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/d

[RFC v2 48/83] Dir: Readdir operation.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA reads the directory by traversing the log and reports the valid dentries. Valid dentris have inode number greater than zero, meaning it's a create dentry. Signed-off-by: Andiry Xu --- fs/nova/dir.c | 153 fs/nova

[RFC v2 01/83] Introduction and documentation of NOVA filesystem.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> NOVA is a log-structured file system tailored for byte-addressable non-volatile memories. It was designed and developed at the Non-Volatile Systems Laboratory in the Computer Science and Engineering Department at the University of California, San Dieg

[RFC v2 00/83] NOVA: a new file system for persistent memory

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> This is the second version of RFC patch series that impements NOVA (NOn-Volatile memory Accelerated file system), a new file system built for PMEM. NOVA's goal is to provide a high performance, production-ready file system tailored for byte-addressab

[RFC v2 01/83] Introduction and documentation of NOVA filesystem.

2018-03-10 Thread Andiry Xu
From: Andiry Xu NOVA is a log-structured file system tailored for byte-addressable non-volatile memories. It was designed and developed at the Non-Volatile Systems Laboratory in the Computer Science and Engineering Department at the University of California, San Diego. Its primary authors

[RFC v2 00/83] NOVA: a new file system for persistent memory

2018-03-10 Thread Andiry Xu
From: Andiry Xu This is the second version of RFC patch series that impements NOVA (NOn-Volatile memory Accelerated file system), a new file system built for PMEM. NOVA's goal is to provide a high performance, production-ready file system tailored for byte-addressable non-volatile memories

[RFC v2 10/83] Add superblock integrity check.

2018-03-10 Thread Andiry Xu
From: Andiry Xu <jix...@cs.ucsd.edu> Repair broken primary superblock with redundant superblock. Signed-off-by: Andiry Xu <jix...@cs.ucsd.edu> --- fs/nova/super.c | 102 1 file changed, 102 insertions(+) diff --git a/fs/nova/

[RFC v2 10/83] Add superblock integrity check.

2018-03-10 Thread Andiry Xu
From: Andiry Xu Repair broken primary superblock with redundant superblock. Signed-off-by: Andiry Xu --- fs/nova/super.c | 102 1 file changed, 102 insertions(+) diff --git a/fs/nova/super.c b/fs/nova/super.c index 552fe5d..e0e38ab

Filebench failure on ramdisk with Ext4-DAX

2015-07-07 Thread Andiry Xu
Hi, I am running into failures when run filebench on ramdisk(/dev/ram0) with Ext4-DAX. The kernel version is 4.0, and I also verified it occurs on 4.2-rc1. The issue reproduction steps: // Set ramdisk size to 2GB # mkfs.ext4 /dev/ram0 # mount -o dax /dev/ram0 /mnt/ramdisk # filebench filebench>

Filebench failure on ramdisk with Ext4-DAX

2015-07-07 Thread Andiry Xu
Hi, I am running into failures when run filebench on ramdisk(/dev/ram0) with Ext4-DAX. The kernel version is 4.0, and I also verified it occurs on 4.2-rc1. The issue reproduction steps: // Set ramdisk size to 2GB # mkfs.ext4 /dev/ram0 # mount -o dax /dev/ram0 /mnt/ramdisk # filebench filebench

<    1   2   3   >