[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Sure. But maybe we need to take more time since he is recently very busy. If this bug doesn't have much affect on you, we could wait. Otherwise, you can test it and apply it in your kernel. After all, it can solve your problem though it may not be the best way. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: In Progress Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Hi Jason, thanks for taking this. I'm happy to test in the coming week or so, we also need to get Tyler's thoughts since he's going to need to sign off too. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: In Progress Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
** Tags added: patch -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: In Progress Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Hello Benjamin Gemmill, I'm done with the patch (see the attachment). It can be applied in the latest linus' tree and 4.8.17 kernel.If you have any idea or advice, please tell me. Thanks. ** Patch added: "0001-ecryptfs-Fix-stat-command-displays-wrong-file-size.patch" https://bugs.launchpad.net/ecryptfs/+bug/1612492/+attachment/4900807/+files/0001-ecryptfs-Fix-stat-command-displays-wrong-file-size.patch ** Changed in: ecryptfs Status: Triaged => In Progress -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: In Progress Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
I tested different kernels and two filesystem (xfs and ext4). It turns out that we cannot read less than value_size (which is 81 in ecryptfs) buffer when doing ecryptfs_read_and_validate_xattr_region(). In ecryptfs_read_and_validate_xattr_region() function, "file_size" array is the buffer we're going to read from xatte region and write. The reasonable solution is read 4096 instead of 16 from xattr region. Any thoughts? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Triaged Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
I'm working on this bug issue, so I reassign to me. The maintainers of eCryptfs are very busy, if they want to write the patch, that would be great! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Triaged Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Tyler, if you are writing the patch, please tell me, I will assign it to you. ** Changed in: ecryptfs Assignee: Tyler Hicks (tyhicks) => Jason Xing (wlxing) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Triaged Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Also verified in 3.3.8 I compile the kernel (3.3.8) on Ubuntu 14.04 and Centos 7-1511. After I mount ecryptfs with ecryptfs_xattr option, it will terminate if I copy some file in it. First time I copy files, it will be killed. Next time I copy files, it will hang and do nothing. Test outputs: $ sudo mount -t ecryptfs secure raw -o ecryptfs_xattr [...] $ cp file.out2 raw Killed I test 3.3.8 kernel on different machines including VM-ware, the outputs are all the same! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Triaged Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
It also affects me. I tested it on 14.04 (64-bit) and 16.04 (64-bit). Verified in 4.2.0-27-generic and 4.8.0-36-generic. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Triaged Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
The bug is in ecryptfs_read_and_validate_xattr_region(). It calls ecryptfs_getxattr_lower(), which is returning -34 (ERANGE). With ecryptfs_getxattr_lower() failing, it means that we can't retrieve the proper file size from the xattr metadata and can't set an an accurate value in the kstat buffer. ** Changed in: ecryptfs Status: Confirmed => Triaged -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Triaged Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Hi Tyler, thanks for commenting on this. I'm trying to get encryptfs going over a remote filesystem, and xattrs are the only practical place to put crypto headers due to file reading speeds: https://www.amazon.com/clouddrive/ And in particular, if you go to read any part of a file, you have to get the whole file: https://developer.amazon.com/public/apis/experience/cloud-drive/content/nodes You can imagine in a directory with many things, or even just a few huge things, an 'ls -al' can make the system start downloading a lot while also appearing to hang. To get ecryptfs working practically on this kind of remote filesystem, we'd have to use xattrs. Do you know what would be involved for a fix? I'd be happy to contribute if you could point me in the right direction. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Confirmed Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Yes, it is caused by the ecryptfs_xattr_metadata option. A workaround would be to not store the crypto metadata in the xattr and use the default of storing the metadata in the file contents. The ecryptfs_xattr_metadata option is not the default and, therefore, is not as widely used/tested. This is a bug that needs to be fixed but I'd recommend not using ecryptfs_xattr_metadata. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Confirmed Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1612492] Re: remounting breaks size reporting and rsync
Verified in 4.8-rc2. ** Changed in: ecryptfs Status: New => Confirmed ** Changed in: ecryptfs Importance: Undecided => Medium ** Changed in: ecryptfs-utils Status: New => Invalid ** Package changed: ubuntu => linux (Ubuntu) ** Changed in: linux (Ubuntu) Importance: Undecided => Low ** Changed in: linux (Ubuntu) Status: New => Confirmed ** Changed in: linux (Ubuntu) Assignee: (unassigned) => Tyler Hicks (tyhicks) ** Changed in: ecryptfs Assignee: (unassigned) => Tyler Hicks (tyhicks) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1612492 Title: remounting breaks size reporting and rsync Status in eCryptfs: Confirmed Status in ecryptfs-utils: Invalid Status in linux package in Ubuntu: Confirmed Bug description: Sorry to resurrect an old bug, it looks like a fix applied a while back may end up breaking rsync. For reference, it looks like the decision in this bug fix was to pass on the encrypted file size on disk to stat calls looking at the unencrypted mount: https://bugs.launchpad.net/ecryptfs/+bug/390833 For reference, to reproduce: mkdir /mnt/raw mkdir /mnt/decrypted dd if=/dev/urandom of=file.out2 bs=1MB count=100 mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted rsync file.out2 /mnt/decrypted/ stat /mnt/decrypted/file.out2 # you get size 1 umount /mnt/decrypted/ mount -t ecryptfs -o ecryptfs_passthrough=n,no_sig_cache,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,ecryptfs_enable_filename_crypto=y,passphrase_passwd=SECRET,ecryptfs_fnek_sig=SECRET /mnt/raw /mnt/decrypted stat /mnt/decrypted/file.out2 # you get size 13840 What this means in practice is any additional rsyncs to that target after remounting will see the file size as different than the original, and issue a full new copy. My use case is ecryptfs over a remote file system, so rsync is no better than cp. Would it be possible to make file size reporting consistent, and reflective of the size of the file as a reader would see it? To manage notifications about this bug go to: https://bugs.launchpad.net/ecryptfs/+bug/1612492/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
