Re: [Gluster-devel] NetBSD regression fixes
On Wed, Mar 18, 2015 at 03:13:32AM -0400, Joseph Fernandes wrote: The smoke for http://review.gluster.org/#/c/9683/ on netbsd fails http://build.gluster.org/job/freebsd-smoke/4066/ This is FreeBSD, the NetBSD builds goes fine. Could you please install sqlite3 and sqlite-devel rpms on the test machines. I believe Justin is the maintainer of this box. I do not have particular skills with FreeBSD. It may look odd since NetBSD and FreeBSD have very close names, but they are really different OS that forked 22 years ago from 386BSD. -- Emmanuel Dreyfus m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] feature/trash and NetBSD
On Wed, Mar 18, 2015 at 11:26:02AM +0530, Anoop C S wrote:
Trash translator creates the original file inside .trashcan directory on
receiving a truncate/ftruncate call and then truncates the file. During
the above mentioned procedure, whether a truncate/ftruncate call is
issued internally?
The kernel sends SETATTR with new file size, and glusterfs turns this
into a truncate call. I have to dig deeper to tell how.
We have started looking into the failures. And I see many instances of
the following error:
du: unknown option -- b
Yes, I had to fix a few shell portability problems
- The -p option for mkdir must be before the directory name
- du -b is not portable. Use ls -l instead.
- test -e file* fails if file* expands to multiple files.
I introduced wildcard_exists() and wildcard_not_exists() to work
this around.
Attached patch adresses the three points.
--
Emmanuel Dreyfus
m...@netbsd.org
diff --git a/tests/features/trash.t b/tests/features/trash.t
index e0b85da..bdb56f3 100755
--- a/tests/features/trash.t
+++ b/tests/features/trash.t
@@ -37,29 +37,35 @@ unlink_op() {
sleep 2
test ! -e $M0/$1
-test -e $M0/.trashcan/$1*
+wildcard_exists $M0/.trashcan/$1*
# remove from trashcan
rm -f $M0/.trashcan/$1*
-test ! -e $M0/.trashcan/$1*
+wildcard_not_exists $M0/.trashcan/$1*
}
truncate_op() {
-
truncate -s 2 $M0/$1
ls $M0/.trashcan/1/2/3 /dev/null
sleep 2
test -e $M0/$1
-test $(du -b $M0/$1 | awk '{print $1}') -eq 2 /dev/null
-test -e $M0/.trashcan/$1*
-test $(du -b $M0/.trashcan/$1*|awk '{print $1}') -eq $2 /dev/null
+test $(ls -l $M0/$1 | awk '{print $5}') -eq 2 /dev/null
+wildcard_exists $M0/.trashcan/$1*
+test $(ls -l $M0/.trashcan/$1*|awk '{print $5}') -eq $2 /dev/null
# truncate from trashcan
truncate -s 1 $M0/.trashcan/$1*
test $(ls $M0/.trashcan/$1* | wc -l) -eq 1
}
+wildcard_exists() {
+test -e $1
+}
+
+wildcard_not_exists() {
+test ! -e $1
+}
# testing glusterd [1-3]
TEST glusterd
@@ -94,7 +100,7 @@ TEST unlink_op file1
TEST truncate_op file2 4
# create files directory hierarchy and check [16]
-mkdir $M0/1/2/3 -p
+mkdir -p $M0/1/2/3
create_files $M0/1/2/3/foo1 $M0/1/2/3/foo2
TEST file_exists 1/2/3/foo1 1/2/3/foo2
@@ -117,13 +123,13 @@ TEST file_exists a/test1 a/test2
# remove from eliminate pattern [22]
rm -f $M0/a/test1
-TEST [ ! -e $M0/.trashcan/a/test1* ]
+TEST wildcard_not_exists $M0/.trashcan/a/test1*
# truncate from eliminate path [23-25]
truncate -s 2 $M0/a/test2
TEST [ -e $M0/a/test2 ]
-TEST [ `du -b $M0/a/test2 | awk '{print $1}'` -eq 2 ]
-TEST [ ! -e $M0/.trashcan/a/test2* ]
+TEST [ `ls -l $M0/a/test2 | awk '{print $5}'` -eq 2 ]
+TEST wildcard_not_exists $M0/.trashcan/a/test2*
# set internal op on [26-27]
TEST $CLI volume set $V0 features.trash-internal-op on
@@ -157,7 +163,7 @@ sleep 3
# check whether rebalance was succesful [38-40]
TEST [ -e $B0/${V0}3/rebal2 ]
-TEST [ -e $B0/${V0}1/.trashcan/internal_op/rebal2* ]
+TEST wildcard_exists $B0/${V0}1/.trashcan/internal_op/rebal2*
TEST stop_vol
# create a replicated volume [41]
@@ -190,21 +196,21 @@ TEST [ -e $B0/${V1}1/self -a -e $B0/${V1}2/self ]
kill `ps aux| grep glusterfsd | awk '{print $2}' | head -1`
sleep 2
rm -f $M1/self
-TEST [ -e $M1/.trashcan/self* ]
+TEST wildcard_exists $M1/.trashcan/self*
# force start the volume and trigger the self-heal manually [56]
TEST $CLI volume start $V1 force
sleep 3
# check for the removed file in trashcan [57]
-TEST [ -e $B0/${V1}1/.trashcan/internal_op/self* -o -e
$B0/${V1}2/.trashcan/internal_op/self* ]
+TEST [ wildcard_exists $B0/${V1}1/.trashcan/internal_op/self* -o
wildcard_exists $B0/${V1}2/.trashcan/internal_op/self* ]
# check renaming of trash directory through cli [58-62]
TEST $CLI volume set $V0 trash-dir abc
TEST $CLI volume start $V0
TEST glusterfs -s $H0 --volfile-id $V0 $M0 --attribute-timeout=0
TEST [ -e $M0/abc -a ! -e $M0/.trashcan ]
-TEST [ -e $B0/${V0}1/abc/internal_op/rebal2* ]
+TEST wildcard_exists $B0/${V0}1/abc/internal_op/rebal2*
sleep 2
# ensure that rename and delete operation on trash directory fails [63-65]
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] feature/trash and NetBSD
On 03/18/2015 02:00 PM, Emmanuel Dreyfus wrote: On Wed, Mar 18, 2015 at 11:26:02AM +0530, Anoop C S wrote: Trash translator creates the original file inside .trashcan directory on receiving a truncate/ftruncate call and then truncates the file. During the above mentioned procedure, whether a truncate/ftruncate call is issued internally? The kernel sends SETATTR with new file size, and glusterfs turns this into a truncate call. I have to dig deeper to tell how. Ok. As of now trash translator doesn't distinguish between extended truncate and reduced truncate. From your explanation and discussion with Vijay, we doubt that it is an extended truncate(truncate with a higher size). We are planning to skip extended truncate from trashing. A patch will be sent out soon to address the problem. If that is the case, then the corresponding (upcoming) patch will solve the issue. We have started looking into the failures. And I see many instances of the following error: du: unknown option -- b Yes, I had to fix a few shell portability problems - The -p option for mkdir must be before the directory name - du -b is not portable. Use ls -l instead. - test -e file* fails if file* expands to multiple files. I introduced wildcard_exists() and wildcard_not_exists() to work this around. Attached patch adresses the three points. Thanks for pointing out the issues. When we have our above mentioned patch merged upstream, we can verify and then you can send this patch for trash.t upstream for review. --Anoop C S. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression in quota-nfs.t
On Wed, Mar 18, 2015 at 10:03:28AM +, Emmanuel Dreyfus wrote: Any idea anyone? Perhaps we could justgot err instead? if (uuid_is_null (local-loc.gfid)) goto err; Indeed, the test passes with this patch: diff --git a/xlators/features/marker/src/marker-quota.c b/xlators/features/marker/src/marker-quota.c index b8d7a77..4f8d057 100644 --- a/xlators/features/marker/src/marker-quota.c +++ b/xlators/features/marker/src/marker-quota.c @@ -1183,7 +1183,8 @@ mq_get_xattr (call_frame_t *frame, void *cookie, xlator_t *this, if (uuid_is_null (local-loc.gfid)) uuid_copy (local-loc.gfid, local-loc.inode-gfid); -GF_UUID_ASSERT (local-loc.gfid); +if (uuid_is_null (local-loc.gfid)) +goto err; STACK_WIND (frame, mq_check_n_set_inode_xattr, FIRST_CHILD(this), FIRST_CHILD(this)-fops-lookup, local-loc, xattr_req); -- Emmanuel Dreyfus m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] NetBSD regression in quota-nfs.t
Hi NetBSD regression has been broken a lot latetly, here is the latest one I investigated: tests/basic/quota-nfs.t It fails on this test: TEST $CLI volume quota $V0 limit-usage / 20MB Because: quota command failed : Failed to find the directory /autobuild/install/var/run/gluster/patchy/. Reason : Socket is not connected And it happens because the glusterfsd daemon crashed on an assertion: #3 0xbb47f4f9 in __assert13 () from /usr/lib/libc.so.12 #4 0xb9b8045d in mq_get_xattr (frame=0xb970a670, cookie=0xb9702ac8, this=0xbb2d7018, op_ret=0, op_errno=0, xdata=0x0) at marker-quota.c:1186 #5 0xbb764152 in default_inodelk_cbk (frame=0xb9702ac8, cookie=0xb9702b58, this=0xbb2d4018, op_ret=0, op_errno=0, xdata=0x0) at defaults.c:1175 #6 0xb9bc1556 in pl_common_inodelk (frame=0xb9702b58, this=0xbb2d3018, volume=0xb9705838 patchy-marker, inode=0xb9702a38, cmd=9, flock=0xb97270f4, loc=0xb9727048, fd=0x0, xdata=0x0) at inodelk.c:804 #7 0xb9bc15db in pl_inodelk (frame=0xb9702b58, this=0xbb2d3018, volume=0xb9705838 patchy-marker, loc=0xb9727048, cmd=9, flock=0xb97270f4, xdata=0x0) at inodelk.c:816 #8 0xbb768f06 in default_inodelk_resume (frame=0xb9702ac8, this=0xbb2d4018, volume=0xb9705838 patchy-marker, loc=0xb9727048, cmd=9, lock=0xb97270f4, xdata=0x0) at defaults.c:1577 #9 0xbb7833c5 in call_resume_wind (stub=0xb9727028) at call-stub.c:2454 #10 0xbb78a411 in call_resume (stub=0xb9727028) at call-stub.c:2841 (gdb) frame 4 #4 0xb9b8045d in mq_get_xattr (frame=0xb970a670, cookie=0xb9702ac8, this=0xbb2d7018, op_ret=0, op_errno=0, xdata=0x0) at marker-quota.c:1186 1186GF_UUID_ASSERT (local-loc.gfid) Any idea anyone? Perhaps we could justgot err instead? if (uuid_is_null (local-loc.gfid)) goto err; -- Emmanuel Dreyfus m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Minutes for Weekly Gluster Community Meeting: 2015-03-18
Thanks for everyone who attended our Weekly Gluster Community Meeting on IRC a few minutes ago. Meeting Minutes: http://meetbot.fedoraproject.org/gluster-meeting/2015-03-18/gluster-meeting.2015-03-18-11.59.html -- Kaleb ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression fixes
On 18 Mar 2015, at 08:21, Emmanuel Dreyfus m...@netbsd.org wrote: On Wed, Mar 18, 2015 at 03:13:32AM -0400, Joseph Fernandes wrote: The smoke for http://review.gluster.org/#/c/9683/ on netbsd fails http://build.gluster.org/job/freebsd-smoke/4066/ This is FreeBSD, the NetBSD builds goes fine. Could you please install sqlite3 and sqlite-devel rpms on the test machines. I believe Justin is the maintainer of this box. It's Harsha (CC'd). + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression in quota-nfs.t
On Wed, Mar 18, 2015 at 10:28:37AM +, Emmanuel Dreyfus wrote: Indeed, the test passes with this patch: And when submitting it I noticed ithat change has laready been done. -- Emmanuel Dreyfus m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] NetBSD regression in quota-nfs.t
Hi Emmanuel, I have submitted another patch: http://review.gluster.org/#/c/9478/ for addressing the spurious failure with quota-nfs.t Thanks, Vijay On Wednesday 18 March 2015 07:40 PM, Emmanuel Dreyfus wrote: On Wed, Mar 18, 2015 at 10:28:37AM +, Emmanuel Dreyfus wrote: Indeed, the test passes with this patch: And when submitting it I noticed ithat change has laready been done. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Query: Trash feature.
On 03/19/2015 10:58 AM, Kotresh Hiremath Ravishankar wrote:
Hi All,
Why is .trashcan directory created at the same level as .glusterfs and
not inside .glusterfs? In glusterfs, crawlers, usually crawl back end
filesystem and ignore .glusterfs. Now in addition, they have to ignore
.trashcan.
As per the design of trash translator, it should be visible from the
mount point. That's why trash directory is not included in .glusterfs.
Moreover trash directory can be identified with a fixed gfid
{---0005}.
--Anoop C S.
Thanks and Regards,
Kotresh H R
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
