[Gluster-devel] Failed tests/basic/uss.t
Hello everyone, I have cloned and built today's GlusterFS. The test tests/basic/uss.t fails. The console log and /var/log/glusterfs/mnt-glusterfs-0 attached. Any ideas? Thanks in advance, Edward.= TEST 75 (line 159): fd_close 5 ok 75 RESULT 75: 0 = TEST 76 (line 162): gluster --mode=script --wignore volume set patchy features.snapshot-directory .history ok 76 RESULT 76: 0 = TEST 77 (line 164): ls /mnt/glusterfs/0/.history ls: cannot access /mnt/glusterfs/0/.history: No such file or directory not ok 77 RESULT 77: 2 ls: cannot access /mnt/glusterfs/0/.history: No such file or directory = TEST 78 (line 168): [ 0 == 4 ] not ok 78 RESULT 78: 1 = TEST 79 (line 170): ls /mnt/glusterfs/0/.history/snap1 ls: cannot access /mnt/glusterfs/0/.history/snap1: No such file or directory not ok 79 RESULT 79: 2 = TEST 80 (line 171): ls /mnt/glusterfs/0/.history/snap2 ls: cannot access /mnt/glusterfs/0/.history/snap2: No such file or directory not ok 80 RESULT 80: 2 = TEST 81 (line 172): ls /mnt/glusterfs/0/.history/snap3 ls: cannot access /mnt/glusterfs/0/.history/snap3: No such file or directory not ok 81 RESULT 81: 2 = TEST 82 (line 173): ls /mnt/glusterfs/0/.history/snap4 ls: cannot access /mnt/glusterfs/0/.history/snap4: No such file or directory not ok 82 RESULT 82: 2 = TEST 83 (line 175): ls /mnt/glusterfs/0/.history/snap3/dir1 ls: cannot access /mnt/glusterfs/0/.history/snap3/dir1: No such file or directory not ok 83 RESULT 83: 2 = TEST 84 (line 176): ls /mnt/glusterfs/0/.history/snap3/dir2 ls: cannot access /mnt/glusterfs/0/.history/snap3/dir2: No such file or directory not ok 84 RESULT 84: 2 = TEST 85 (line 178): ls /mnt/glusterfs/0/.history/snap4/dir1 ls: cannot access /mnt/glusterfs/0/.history/snap4/dir1: No such file or directory not ok 85 RESULT 85: 2 = TEST 86 (line 179): ls /mnt/glusterfs/0/.history/snap4/dir2 ls: cannot access /mnt/glusterfs/0/.history/snap4/dir2: No such file or directory not ok 86 RESULT 86: 2 = TEST 87 (line 181): ls /mnt/glusterfs/0/dir1/.history/ ls: cannot access /mnt/glusterfs/0/dir1/.history/: No such file or directory not ok 87 RESULT 87: 2 = TEST 88 (line 182): ! ls /mnt/glusterfs/0/dir1/.history/snap1 ok 88 RESULT 88: 0 = TEST 89 (line 183): ! ls /mnt/glusterfs/0/dir2/.history/snap2 ok 89 RESULT 89: 0 = TEST 90 (line 184): ls /mnt/glusterfs/0/dir1/.history/snap3 ls: cannot access /mnt/glusterfs/0/dir1/.history/snap3: No such file or directory not ok 90 RESULT 90: 2 = TEST 91 (line 185): ls /mnt/glusterfs/0/dir2/.history/snap4 ls: cannot access /mnt/glusterfs/0/dir2/.history/snap4: No such file or directory not ok 91 RESULT 91: 2 = TEST 92 (line 187): fd1=4 ok 92 RESULT 92: 0 = TEST 93 (line 188): fd_open 4 r /mnt/glusterfs/0/.history/snap1/file1 ./tests/basic/../fileio.rc: line 21: /mnt/glusterfs/0/.history/snap1/file1: No such file or directory not ok 93 RESULT 93: 1 = TEST 94 (line 189): fd_cat 4 ./tests/basic/../fileio.rc: line 35: 4: Bad file descriptor not ok 94 RESULT 94: 1 = TEST 95 (line 192): fd2=4 ok 95 RESULT 95: 0 = TEST 96 (line 193): ! fd_open 4 w /mnt/glusterfs/0/.history/snap1/file2 ok 96 RESULT 96: 0 = TEST 97 (line 196): ! stat /mnt/glusterfs/0/.history/snap1/.history ok 97 RESULT 97: 0 = TEST 98 (line 199): ! mkdir /mnt/glusterfs/0/.history/new ok 98 RESULT 98: 0 = TEST 99 (line 200): ! touch /mnt/glusterfs/0/.history/snap2/other ok 99 RESULT 99: 0 = TEST 100 (line 202): fd3=4 ok 100 RESULT 100: 0 = TEST 101 (line 203): fd_open 4 r /mnt/glusterfs/0/dir1/.history/snap3/foo1 ok 101 RESULT 101: 0 = TEST 102 (line 205): fd_cat 4 ok 102 RESULT 102: 0 = TEST 103 (line 207): fd_close 4 ok 103 RESULT 103: 0 = TEST 104 (line 208): fd_close 4 ok 104 RESULT 104: 0 = TEST 105 (line 209): fd_close 4 ok 105 RESULT 105: 0 = TEST 106 (line 214): ls /mnt/nfs/0/.history ok 106 RESULT 106: 0 = TEST 107 (line 218): [ 4 == 4 ] ok 107 RESULT 107: 0 = TEST 108 (line 220): ls -l /mnt/nfs/0/.history/snap1 ok 108 RESULT 108: 0 = TEST 109 (line 221): ls -l /mnt/nfs/0/.history/snap2 ok 109 RESULT 109: 0 = TEST 110 (line 222): ls -l /mnt/nfs/0/.history/snap3 ok 110 RESULT 110: 0 = TEST 111 (line 223):
Re: [Gluster-devel] question on crypt xlator
Hello Emmanuel,
Could you please replace the file ./tests/encryption/crypt.t
with the attached one, run the last one, and report about results?
Thanks,
Edward.
On Fri, 10 Oct 2014 02:00:28 +0200
m...@netbsd.org (Emmanuel Dreyfus) wrote:
Edward Shishkin edw...@redhat.com wrote:
1) What is the number of the failed step?
Here is the log. The fact it happens after umount/remount suggests the
metadata used changed a bit:
Running tests in file ./tests/encryption/crypt.t
[23:52:46] ./tests/encryption/crypt.t .. =
TEST 1 (line 8): glusterd
[23:52:46] ./tests/encryption/crypt.t .. 1/? RESULT 1: 0
=
TEST 2 (line 9): pidof glusterd
RESULT 2: 0
=
TEST 3 (line 12): gluster --mode=script --wignore volume create patchy
bacasable.example.net:/d/backends/patchy1
[23:52:46] ./tests/encryption/crypt.t .. 3/? RESULT 3: 0
=
TEST 4 (line 13): patchy volinfo_field patchy Volume Name
RESULT 4: 0
=
TEST 5 (line 14): Created volinfo_field patchy Status
RESULT 5: 0
=
TEST 6 (line 15): 1 brick_count patchy
[23:52:46] ./tests/encryption/crypt.t .. 6/? RESULT 6: 0
=
TEST 7 (line 19): gluster --mode=script --wignore volume set patchy
performance.quick-read off
RESULT 7: 0
=
TEST 8 (line 20): off volinfo_field patchy performance.quick-read
[23:52:46] ./tests/encryption/crypt.t .. 8/? RESULT 8: 0
=
TEST 9 (line 21): gluster --mode=script --wignore volume set patchy
performance.write-behind off
RESULT 9: 0
=
TEST 10 (line 22): off volinfo_field patchy performance.write-behind
[23:52:46] ./tests/encryption/crypt.t .. 10/? RESULT 10: 0
=
TEST 11 (line 23): gluster --mode=script --wignore volume set patchy
performance.open-behind off
RESULT 11: 0
=
TEST 12 (line 24): off volinfo_field patchy performance.open-behind
RESULT 12: 0
=
TEST 13 (line 27): gluster --mode=script --wignore volume set patchy
encryption on
[23:52:46] ./tests/encryption/crypt.t .. 13/? RESULT 13: 0
=
TEST 14 (line 28): on volinfo_field patchy features.encryption
RESULT 14: 0
=
TEST 15 (line 31): gluster --mode=script --wignore volume set patchy
encryption.master-key /tmp/patchy-master-key
[23:52:46] ./tests/encryption/crypt.t .. 15/? RESULT 15: 0
=
TEST 16 (line 38): gluster --mode=script --wignore volume start patchy
[23:52:46] ./tests/encryption/crypt.t .. 16/? RESULT 16: 0
=
TEST 17 (line 39): Started volinfo_field patchy Status
RESULT 17: 0
=
TEST 18 (line 42): glusterfs --attribute-timeout=0 --entry-timeout=0
--volfile-server=bacasable.example.net --volfile-id=patchy
/mnt/glusterfs/0
RESULT 18: 0
=
TEST 19 (line 48): ./tests/encryption/frag /mnt/glusterfs/0/testfile
/tmp/patchy-goodfile 262144 500
[23:52:46] ./tests/encryption/crypt.t .. 19/? RESULT 19: 0
=
TEST 20 (line 52): ln /mnt/glusterfs/0/testfile
/mnt/glusterfs/0/testfile-link
RESULT 20: 0
=
TEST 21 (line 53): mv /mnt/glusterfs/0/testfile
/mnt/glusterfs/0/testfile-renamed
RESULT 21: 0
=
TEST 22 (line 54): ln -s /mnt/glusterfs/0/testfile-link
/mnt/glusterfs/0/testfile-symlink
RESULT 22: 0
=
TEST 23 (line 55): rm -f /mnt/glusterfs/0/testfile-renamed
RESULT 23: 0
=
TEST 24 (line 58): Y force_umount /mnt/glusterfs/0
RESULT 24: 0
=
TEST 25 (line 59): glusterfs --volfile-server=bacasable.example.net
--volfile-id=patchy /mnt/glusterfs/0
RESULT 25: 0
=
TEST 26 (line 61): diff -u /mnt/glusterfs/0/testfile-symlink
/tmp/patchy-goodfile
diff: /mnt/glusterfs/0/testfile-symlink: Invalid argument
not ok 26
RESULT 26: 2
2) What endianess does your machine have?
Little endian (i386)
#!/bin/bash
. $(dirname $0)/../include.rc
. $(dirname $0)/../volume.rc
cleanup;
TEST glusterd
TEST pidof glusterd
## Create a volume with one brick
TEST $CLI volume create $V0 $H0:$B0/${V0}1;
EXPECT $V0 volinfo_field $V0 'Volume Name';
EXPECT 'Created' volinfo_field $V0 'Status';
EXPECT '1' brick_count $V0
## Turn off performance translators
TEST $CLI volume set $V0 performance.quick-read off
EXPECT 'off' volinfo_field $V0 'performance.quick-read'
TEST $CLI volume set $V0 performance.write-behind off
EXPECT 'off' volinfo_field $V0 'performance.write-behind'
TEST $CLI volume set $V0 performance.open-behind off
EXPECT 'off' volinfo_field $V0 'performance.open-behind'
## Turn on crypt xlator by setting features.encryption to on
TEST $CLI volume set $V0 encryption on
EXPECT 'on' volinfo_field $V0
Re: [Gluster-devel] question on crypt xlator
On Mon, 6 Oct 2014 12:19:44 + Emmanuel Dreyfus m...@netbsd.org wrote: Hi Hello Emmanuel, crtypt.t gives me this on a symlink after umount/remount: [2014-10-06 09:28:17.199230] E [metadata.c:534:open_format_v1] 0-crypt: EMTD verification failed [2014-10-06 09:28:17.200028] W [crypt.c:74:get_crypt_inode_info] 0-patchy-crypt: Can not get inode info I understand this is about file metdata signature, and I suspect something subbtle must change over remounts in NetBSD, but what? What crypt xlator uses for metadata? Reading the sources gives no obvious answer. We cipher some per-file encryption attributes in the AEAD (GCM) mode. EMTD verification failed means failed authentication part of AEAD mode (EMTD stands for Encrypted part of MeTaData). Normally this message indicates tampering, that is the pair (encrypted_attributes, MAC) received from the non-trusted server differs from the original one that was created on the trusted client machine. However, in your case it is most likely because of a bug in the crypt translator. TBH, I haven't tested this on NetBSD. We'll try to narrow down the problem. For now I have the following questions: 1) What is the number of the failed step? 2) What endianess does your machine have? Thanks for the report, Edward. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Transparent encryption in GlusterFS: Implications on manageability
Hi all, Unfortunately it is impossible to validate non-trusted volfiles using existing glusterfs options. Semantic and format of values passed by the --xlator-option don't allow to deliver trusted values without compromises with security. So I have added a new --secure-xlator-option, Please, review: review.gluster.org/8657 Thanks, Edward. On Wed, 13 Aug 2014 12:26:29 -0700 Anand Avati av...@gluster.org wrote: +1 for all the points. On Wed, Aug 13, 2014 at 11:22 AM, Jeff Darcy jda...@redhat.com wrote: I.1 Generating the master volume key Master volume key should be generated by user on the trusted machine. Recommendations on master key generation provided at section 6.2 of the manpages [1]. Generating of master volume key is in user's competence. That was fine for an initial implementation, but it's still the single largest obstacle to adoption of this feature. Looking forward, we need to provide full CLI support for generating keys in the necessary format, specifying their location, etc. I.2 Location of the master volume key when mounting a volume At mount time the crypt translator searches for a master volume key on the client machine at the location specified by the respective translator option. If there is no any key at the specified location, or the key at specified location is in improper format, then mount will fail. Otherwise, the crypt translator loads the key to its private memory data structures. Location of the master volume key can be specified at volume creation time (see option master-key, section 6.7 of the man pages [1]). However, this option can be overridden by user at mount time to specify another location, see section 7 of manpages [1], steps 6, 7, 8. Again, we need to improve on this. We should support this as a volume or mount option in its own right, not rely on the generic --xlator-option mechanism. Adding options to mount.glusterfs isn't hard. Alternatively, we could make this look like a volume option settable once through the CLI, even though the path is stored locally on the client. Or we could provide a separate special-purpose command/script, which again only needs to be run once. It would even be acceptable to treat the path to the key file (not its contents!) as a true volume option, stored on the servers. Any of these would be better than requiring the user to understand our volfile format and construction so that they can add the necessary option by hand. II. Check graph of translators on your client machine after mount! During mount your client machine receives configuration info from the non-trusted server. In particular, this info contains the graph of translators, which can be subjected to tampering, so that encryption won't be invoked for your volume at all. So it is highly important to verify this graph. After successful mount make sure that the graph of translators contains the crypt translator with proper options (see FAQ#1, section 11 of the manpages [1]). It is important to verify the graph, but not by poking through log files and not without more information about what to look for. So we got a volfile that includes the crypt translator, with some options. The *code* should ensure that the master-key option has the value from the command line or local config, and not some other. If we have to add special support for this in otherwise-generic graph initialization code, that's fine. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Transparent encryption in GlusterFS: Implications on manageability
Hello everyone, In this thread we'll consider in details all issues related to the management aspects of the feature Transparent encryption in GlusterFS. We assume that the reader is familiar with the basic concepts of the feature Transparent encryption in GlusterFS. See, for example, slides 1-7 of the design document [2]. Comments, questions, any your experience are welcome. Thanks, Edward. Transparent encryption in GlusterFS: Implications on manageability I. Managing master volume keys Any user of the feature Transparent encryption in GlusterFS will need to manage master volume keys (see man pages [1], section 6.2). I.1 Generating the master volume key Master volume key should be generated by user on the trusted machine. Recommendations on master key generation provided at section 6.2 of the manpages [1]. Generating of master volume key is in user's competence. I.2 Location of the master volume key when mounting a volume At mount time the crypt translator searches for a master volume key on the client machine at the location specified by the respective translator option. If there is no any key at the specified location, or the key at specified location is in improper format, then mount will fail. Otherwise, the crypt translator loads the key to its private memory data structures. Location of the master volume key can be specified at volume creation time (see option master-key, section 6.7 of the man pages [1]). However, this option can be overridden by user at mount time to specify another location, see section 7 of manpages [1], steps 6, 7, 8. I.3 Retention of master volume keys between mount sessions After successful mount the file with master volume key can be removed from the client machine: the crypt translator don't need it for _this_ mount session anymore: it will use the key installed in the private memory data structures of the crypt translator. IMPORTANT: next mount session will require the master volume key again, so if you remove the file with master volume key after the mount command, make sure you have stored it in the safe place for the next mount session. Master volume key is a private secret key, so it is highly important to provide its proper storage (on trusted machines, trusted media, trusted network, etc). Otherwise it can be easily compromised. Keeping the master volume key between mount sessions is in user's competence. If user has more than one encrypted volume, he should maintain the mapping volume - master-volume-key. Maintenance of such mapping is in user's competence. I.4 Don't use the same master volume key for different Gluster volumes! Different volumes should be encrypted with different master volume keys. Using the same master volume key for different volumes is highly not recommended because of security reasons. I.5 Using different master volume keys for the same volume It is possible to mount a volume with a different master volume key (see FAQ #4, section 11 of manpages [1]). In such mount session content of files created with different master volume key will be inaccessible. In particular, the crypt translator will refuse to open files encrypted with different master volume key. However, it will be possible to create new files on this volume and access them in the mount sessions with respective master volume key. In this case instead of simple mapping volume - master-volume-key user will need to maintain mapping file - master-volume-key, which is more complicated. Maintenance of this mapping is in user's competence. I.6 Compromised master volume key If your master volume key has been compromised because of some reasons, or you suspect, that it has been compromised, then the whole volume should be re-encrypted with the newly generated master volume key. The common way is to create a new empty encrypted GlusterFS volume, mount it with the new master volume key (see the manpages [1], section 7), copy the old volume, mounted with compromised key to the new one, mounted with the new key (for example, using cp(1) command. After successful copy overwrite the content of all regular files of your old volume with zeros. After this delete the old volume as usual. Don't use the compromised master volume key anymore. II. Check graph of translators on your client machine after mount! During mount your client machine receives configuration info from the non-trusted server. In particular, this info contains the graph of translators, which can be subjected to tampering, so that encryption won't be invoked for your volume at all. So it is highly important to verify this graph. After successful mount make sure that the graph of translators contains the crypt translator with proper options (see FAQ#1, section 11 of the manpages [1]). III. Changing the
[Gluster-devel] Transparent encryption and authentication in distributed systems with non-trusted servers
Hello everyone, Here we provide some basic backgrounds in addition to the manpages at http://www.gluster.org/community/documentation/index.php?title=Features/disk-encryption Comments, questions, any your experience are welcome. Thanks, Edward. Transparent encryption and authentication in distributed systems with non-trusted servers Distributed systems impose tighter requirements to at-rest encryption. This is because your encrypted data will be stored on servers, which are de facto untrusted. In particular, your private encrypted data can be subjected to analysis and tampering, which eventually will lead to its revealing, if it is not properly protected. Specifically, usually it is not enough to just encrypt data. In distributed systems serious protection of your personal data is possible only in conjunction with a special process, which is called authentication. GlusterFS provides such enhanced service: In GlusterFS encryption is enhanced with authentication. Currently we provide protection from silent tampering. This is a kind of tampering which is hard to detect, because it doesn't break POSIX compliance. Specifically, we protect encryption-specific file's metadata which includes unique file's object id (GFID), cipher algorithm id, cipher block size and other attributes used by the encryption process. Restrictions 1. We encrypt only file content. The feature of transparent encryption doesn't protect file names: they are neither encrypted, nor verified. Protection of file names is not so critical as protection of encryption-specific file's metadata: any attacks based on tampering file names will break POSIX compliance and result in massive corruption, which is easy to detect. 2. The feature of transparent encryption is not supported in NFS mounts of GlusterFS volumes: NFS's file handles introduce security issues, which are hard to resolve. 3. The feature of transparent encryption is incompatible with GlusterFS performance translators (quick-read, write-behind and open-behind). I. Trusted and non-trusted machines Suppose you are a user of this feature (transparent encryption and authentication in GlusterFS). You qualify every machine as either trusted, or non-trusted. Examples: I.1 Your personal laptop, which is under your supervision is trusted machine. I.2 Remote GlusterFS servers, which are not under your supervision are non-trusted machines. They are managed by a good admin, but you don't trust him your private data. I.3 Clouds are important example of a set of non-trusted machines: you don't know what is going on there at all. II. Trusted and non-trusted objects Every machine contains objects (in the RAM, disks, registers, etc). All objects of every non-trusted machine are non-trusted by definition. Trusted machine contains both type of objects (trusted and non-trusted). Sources of non-trusted objects on your trusted machine: . non-trusted media; . non-trusted network; . social engineering; . etc. Sources of trusted objects on your trusted machine: . trusted media; . trusted network; . process of verification of non-trusted objects (authentication). Examples: II.1 email that you have received without any checks is non-trusted object; II.2 email with properly verified digital signature is trusted object; II.3 Your secret key properly generated on your trusted machine, or retrieved from trusted media is trusted object. II.4 You wanted to look at user accounts on your trusted local machine. The string /etc/passwd that you have passed to the open(2) is trusted object. II.5 Someone you don't know asked you to check if you have a file /foo/bar on your trusted local machine. The string /foo/bar, that you have passed to readdir(2) is non-trusted object. II.6 Encrypted content of any regular file received from the non-trusted GlusterFS servers before processing by the crypt translator is non-trusted object. II.7 Decrypted content of your regular file after successful processing by the crypt translator is trusted object. II.8 List of file names provided by ls(1) for your mounted encrypted GlusterFS volume is non-trusted object (see subsection 1 of the Restrictions above). Status of some objects on your trusted machine can be changed from non-trusted to trusted by a special process, which is called authentication. Authentication includes creating/checking a special MAC (Message Authentication Code) for every object that you will want to verify after its storing on the non-trusted machines. III. Encryption and authentication in GlusterFS In GlusterFS a special translator (encryption/crypt) is responsible for both, encryption and authentication. The crypt translator works only on trusted client machines. Data Encryption We encrypt file content by the AES cipher algorithm with XTS
Re: [Gluster-devel] spurios failures in tests/encryption/crypt.t
On Wed, 21 May 2014 00:06:22 -0700 Anand Avati av...@gluster.org wrote: On Tue, May 20, 2014 at 10:54 PM, Pranith Kumar Karampuri pkara...@redhat.com wrote: - Original Message - From: Anand Avati av...@gluster.org To: Pranith Kumar Karampuri pkara...@redhat.com Cc: Edward Shishkin edw...@redhat.com, Gluster Devel gluster-devel@gluster.org Sent: Wednesday, May 21, 2014 10:53:54 AM Subject: Re: [Gluster-devel] spurios failures in tests/encryption/crypt.t There are a few suspicious things going on here.. On Tue, May 20, 2014 at 10:07 PM, Pranith Kumar Karampuri pkara...@redhat.com wrote: hi, crypt.t is failing regression builds once in a while and most of the times it is because of the failures just after the remount in the script. TEST rm -f $M0/testfile-symlink TEST rm -f $M0/testfile-link Both of these are failing with ENOTCONN. I got a chance to look at the logs. According to the brick logs, this is what I see: [2014-05-17 05:43:43.363979] E [posix.c:2272:posix_open] 0-patchy-posix: open on /d/backends/patchy1/testfile-symlink: Transport endpoint is not connected posix_open() happening on a symlink? This should NEVER happen. glusterfs itself should NEVER EVER by triggering symlink resolution on the server. In this case, for whatever reason an open() is attempted on a symlink, and it is getting followed back onto gluster's own mount point (test case is creating an absolute link). So first find out: who is triggering fop-open() on a symlink. Fix the caller. Next: add a check in posix_open() to fail with ELOOP or EINVAL if the inode is a symlink. I think I understood what you are saying. Open call for symlink on fuse mount lead to an open call again for the target on the same fuse mount. It's not that simple. The client VFS is intelligent enough to resolve symlinks and send open() only on non-symlinks. And the test case script was doing an obvious unlink() (TEST rm -f filename), so it was not initiated by an open() attempt in the first place. My guess is that some xlator (probably crypt?) is doing an open() on an inode Ah, it is quite possible, that it is the crypt.. I'll take a look. Thanks for the hint, I stupidly increased the testcases without chances to reproduce the problem.. and that is going through unchecked in posix. It is a bug in both the caller and posix, but the onus/responsibility is on posix to disallow open() on anything but regular files (even open() on character or block devices should not happen in posix). Which lead to deadlock :). That is why we disallow opens on symlink in gluster? That's not just why open on symlink is disallowed in gluster, it is a more generic problem of following symlinks in general inside gluster. Symlink resolution must strictly happen only in the outermost VFS. Following symlinks inside the filesystem is not only an invalid operation, but can lead to all kinds of deadlocks, security holes (what if you opened a symlink which points to /etc/passwd, should it show the contents of the client machine's /etc/passwd or the server? Now what if you wrote to the file through the symlink? etc. you get the idea..) and wrong/weird/dangerous behaviors. This is not just related to following symlinks, even open()ing special devices.. e.g if you create a char device file with major/minor number of an audio device and wrote pcm data into it, should it play music on the client machine or in the server machine? etc. The summary is, following symlinks or opening non-regular files is VFS/client operation and are invalid operations in a filesystem context. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] spurios failures in tests/encryption/crypt.t
On Sat, 17 May 2014 04:28:45 -0400 (EDT) Pranith Kumar Karampuri pkara...@redhat.com wrote: hi, crypt.t is failing regression builds once in a while and most of the times it is because of the failures just after the remount in the script. TEST rm -f $M0/testfile-symlink TEST rm -f $M0/testfile-link Both of these are failing with ENOTCONN. I got a chance to look at the logs. According to the brick logs, this is what I see: [2014-05-17 05:43:43.363979] E [posix.c:2272:posix_open] 0-patchy-posix: open on /d/backends/patchy1/testfile-symlink: Transport endpoint is not connected This is the very first time I saw posix failing with ENOTCONN. Do we have these bricks on some other network mounts? I wonder why it fails with ENOTCONN. I also see that it happens right after a call_bail on the mount. Pranith Hello. OK, I'll try to reproduce it. Thanks for the report! Edward. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
