date:20160723

Re: [Gluster-devel] readdir() harmful in threaded code

2016-07-23 Thread Pranith Kumar Karampuri

On Sat, Jul 23, 2016 at 8:02 PM, Emmanuel Dreyfus  wrote:

> Pranith Kumar Karampuri  wrote:
>
> > So should we do readdir() with external locks for everything instead?
>
> readdir() with a per-directory lock is safe. However, it may come with a
> performance hit in some scenarios, since two threads cannot read the
> same directory at once. But I am not sure it can happen in GlusterFS.
>
> I am a bit disturbed by readdir_r() being planned for deprecation. The
> Open Group does not say that, or I missed it:
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir.html


I will wait for more people to comment on this. Let us see what they think
as well.


>
>
> --
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

2016-07-23 Thread Pranith Kumar Karampuri

Thanks Atin

On Sat, Jul 23, 2016 at 7:29 PM, Atin Mukherjee  wrote:

>
>
> On Saturday 23 July 2016, Pranith Kumar Karampuri 
> wrote:
>
>> If someone could give +1 on 3.7 backport
>> http://review.gluster.org/#/c/14991, I can merge the patch. Then we can
>> start rebasing may be?
>>
>
> Merged!
>
>
>>
>> On Sat, Jul 23, 2016 at 12:23 PM, Atin Mukherjee 
>> wrote:
>>
>>> AFAIK, an explicit rebase is required.
>>>
>>>
>>> On Saturday 23 July 2016, Pranith Kumar Karampuri 
>>> wrote:
>>>


 On Sat, Jul 23, 2016 at 10:17 AM, Nithya Balachandran <
 nbala...@redhat.com> wrote:

>
>
> On Sat, Jul 23, 2016 at 9:45 AM, Nithya Balachandran <
> nbala...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 9:07 PM, Pranith Kumar Karampuri <
>> pkara...@redhat.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 8:12 PM, Pranith Kumar Karampuri <
>>> pkara...@redhat.com> wrote:
>>>
 I am playing with the following diff, let me see.

 diff --git a/tests/volume.rc b/tests/volume.rc
 index 331a802..b288508 100644
 --- a/tests/volume.rc
 +++ b/tests/volume.rc
 @@ -579,7 +579,9 @@ function num_graphs
  function get_aux()
  {
  ##Check if a auxiliary mount is there
 -df -h 2>&1 | sed 's#/build/install##' | grep -e
 "[[:space:]]/run/gluster/${V0}$" -e 
 "[[:space:]]/var/run/gluster/${V0}$" -
 +local rundir=$(gluster --print-statedumpdir)
 +local pid=$(cat ${rundir}/${V0}.pid)
 +pidof glusterfs 2>&1 | grep -w $pid

  if [ $? -eq 0 ]
  then

>>>
>>> Based on what I saw in code, this seems to get the job done.
>>> Comments welcome:
>>> http://review.gluster.org/14988
>>>
>>>
>> Nice work Pranith :)
>> All, once this is backported to release-3.7, any patches on
>> release-3.7 patches will need to be rebased so they will pass the NetBSD
>> regression.
>>
>
> I am suddenly confused about this - will the patches need to be
> rebased or with the next run automatically include the changes once
> Pranith's fix is merged?
>

 May be someone more knowledgeable about this should confirm this, but
 at least from the build-log, I don't see any rebase command being executed
 with origin/master:

 *04:07:36* Triggered by Gerrit: http://review.gluster.org/13762*04:07:36* 
 Building remotely on slave26.cloud.gluster.org 
  
 (smoke_tests rackspace_regression_2gb glusterfs-devrpms) in workspace 
 /home/jenkins/root/workspace/rackspace-regression-2GB-triggered*04:07:36*  
 > git rev-parse --is-inside-work-tree # timeout=10*04:07:36* Fetching 
 changes from the remote Git repository*04:07:36*  > git config 
 remote.origin.url git://review.gluster.org/glusterfs.git # 
 timeout=10*04:07:36* Fetching upstream changes from 
 git://review.gluster.org/glusterfs.git*04:07:36*  > git --version # 
 timeout=10*04:07:36*  > git -c core.askpass=true fetch --tags --progress 
 git://review.gluster.org/glusterfs.git refs/changes/62/13762/4*04:07:44*  
 > git rev-parse 838b5c34127edd0450b0449e38f075f56056f2c7^{commit} # 
 timeout=10*04:07:44* Checking out Revision 
 838b5c34127edd0450b0449e38f075f56056f2c7 (master)*04:07:44*  > git config 
 core.sparsecheckout # timeout=10*04:07:44*  > git checkout -f 
 838b5c34127edd0450b0449e38f075f56056f2c7*04:07:45*  > git rev-parse 
 FETCH_HEAD^{commit} # timeout=10*04:07:45*  > git rev-list 
 8cbee639520bf4631ce658e2da9b4bc3010d2eaa # timeout=10*04:07:45*  > git tag 
 -a -f -m Jenkins Build #22315 
 jenkins-rackspace-regression-2GB-triggered-22315 # timeout=10




>

 On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran <
 nbala...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran <
>> nbala...@redhat.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy 
>>> wrote:
>>>
 > I attempted to get us more space on NetBSD by creating a new
 partition called
 > /data and putting /build as a symlink to /data/build. This
 has caused
 > problems
 > with tests/basic/quota.t. It's marked as bad for master, but
 not for
 > release-3.7. This is possibly because we have a hard-coded
 grep for
 > /build/install against df -h.

Re: [Gluster-devel] readdir() harmful in threaded code

2016-07-23 Thread Emmanuel Dreyfus

Pranith Kumar Karampuri  wrote:

> So should we do readdir() with external locks for everything instead?

readdir() with a per-directory lock is safe. However, it may come with a
performance hit in some scenarios, since two threads cannot read the
same directory at once. But I am not sure it can happen in GlusterFS.

I am a bit disturbed by readdir_r() being planned for deprecation. The
Open Group does not say that, or I missed it:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir.html

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

2016-07-23 Thread Atin Mukherjee

On Saturday 23 July 2016, Pranith Kumar Karampuri 
wrote:

> If someone could give +1 on 3.7 backport
> http://review.gluster.org/#/c/14991, I can merge the patch. Then we can
> start rebasing may be?
>

Merged!


>
> On Sat, Jul 23, 2016 at 12:23 PM, Atin Mukherjee  > wrote:
>
>> AFAIK, an explicit rebase is required.
>>
>>
>> On Saturday 23 July 2016, Pranith Kumar Karampuri > > wrote:
>>
>>>
>>>
>>> On Sat, Jul 23, 2016 at 10:17 AM, Nithya Balachandran <
>>> nbala...@redhat.com> wrote:
>>>


 On Sat, Jul 23, 2016 at 9:45 AM, Nithya Balachandran <
 nbala...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 9:07 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 8:12 PM, Pranith Kumar Karampuri <
>> pkara...@redhat.com> wrote:
>>
>>> I am playing with the following diff, let me see.
>>>
>>> diff --git a/tests/volume.rc b/tests/volume.rc
>>> index 331a802..b288508 100644
>>> --- a/tests/volume.rc
>>> +++ b/tests/volume.rc
>>> @@ -579,7 +579,9 @@ function num_graphs
>>>  function get_aux()
>>>  {
>>>  ##Check if a auxiliary mount is there
>>> -df -h 2>&1 | sed 's#/build/install##' | grep -e
>>> "[[:space:]]/run/gluster/${V0}$" -e 
>>> "[[:space:]]/var/run/gluster/${V0}$" -
>>> +local rundir=$(gluster --print-statedumpdir)
>>> +local pid=$(cat ${rundir}/${V0}.pid)
>>> +pidof glusterfs 2>&1 | grep -w $pid
>>>
>>>  if [ $? -eq 0 ]
>>>  then
>>>
>>
>> Based on what I saw in code, this seems to get the job done. Comments
>> welcome:
>> http://review.gluster.org/14988
>>
>>
> Nice work Pranith :)
> All, once this is backported to release-3.7, any patches on
> release-3.7 patches will need to be rebased so they will pass the NetBSD
> regression.
>

 I am suddenly confused about this - will the patches need to be rebased
 or with the next run automatically include the changes once Pranith's fix
 is merged?

>>>
>>> May be someone more knowledgeable about this should confirm this, but at
>>> least from the build-log, I don't see any rebase command being executed
>>> with origin/master:
>>>
>>> *04:07:36* Triggered by Gerrit: http://review.gluster.org/13762*04:07:36* 
>>> Building remotely on slave26.cloud.gluster.org 
>>>  (smoke_tests 
>>> rackspace_regression_2gb glusterfs-devrpms) in workspace 
>>> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered*04:07:36*  
>>> > git rev-parse --is-inside-work-tree # timeout=10*04:07:36* Fetching 
>>> changes from the remote Git repository*04:07:36*  > git config 
>>> remote.origin.url git://review.gluster.org/glusterfs.git # 
>>> timeout=10*04:07:36* Fetching upstream changes from 
>>> git://review.gluster.org/glusterfs.git*04:07:36*  > git --version # 
>>> timeout=10*04:07:36*  > git -c core.askpass=true fetch --tags --progress 
>>> git://review.gluster.org/glusterfs.git refs/changes/62/13762/4*04:07:44*  > 
>>> git rev-parse 838b5c34127edd0450b0449e38f075f56056f2c7^{commit} # 
>>> timeout=10*04:07:44* Checking out Revision 
>>> 838b5c34127edd0450b0449e38f075f56056f2c7 (master)*04:07:44*  > git config 
>>> core.sparsecheckout # timeout=10*04:07:44*  > git checkout -f 
>>> 838b5c34127edd0450b0449e38f075f56056f2c7*04:07:45*  > git rev-parse 
>>> FETCH_HEAD^{commit} # timeout=10*04:07:45*  > git rev-list 
>>> 8cbee639520bf4631ce658e2da9b4bc3010d2eaa # timeout=10*04:07:45*  > git tag 
>>> -a -f -m Jenkins Build #22315 
>>> jenkins-rackspace-regression-2GB-triggered-22315 # timeout=10
>>>
>>>
>>>
>>>

>>>
>>> On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran <
>>> nbala...@redhat.com> wrote:
>>>


 On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
 pkara...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran <
> nbala...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy 
>> wrote:
>>
>>> > I attempted to get us more space on NetBSD by creating a new
>>> partition called
>>> > /data and putting /build as a symlink to /data/build. This has
>>> caused
>>> > problems
>>> > with tests/basic/quota.t. It's marked as bad for master, but
>>> not for
>>> > release-3.7. This is possibly because we have a hard-coded
>>> grep for
>>> > /build/install against df -h.
>>>
>>> For the benefit of anyone else looking at this, the grep
>>> actually seems to be
>>> in volume.rc and not

Re: [Gluster-devel] 3.7 regressions on NetBSD

2016-07-23 Thread Pranith Kumar Karampuri

If someone could give +1 on 3.7 backport http://review.gluster.org/#/c/14991,
I can merge the patch. Then we can start rebasing may be?

On Sat, Jul 23, 2016 at 12:23 PM, Atin Mukherjee 
wrote:

> AFAIK, an explicit rebase is required.
>
>
> On Saturday 23 July 2016, Pranith Kumar Karampuri 
> wrote:
>
>>
>>
>> On Sat, Jul 23, 2016 at 10:17 AM, Nithya Balachandran <
>> nbala...@redhat.com> wrote:
>>
>>>
>>>
>>> On Sat, Jul 23, 2016 at 9:45 AM, Nithya Balachandran <
>>> nbala...@redhat.com> wrote:
>>>


 On Fri, Jul 22, 2016 at 9:07 PM, Pranith Kumar Karampuri <
 pkara...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 8:12 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
>> I am playing with the following diff, let me see.
>>
>> diff --git a/tests/volume.rc b/tests/volume.rc
>> index 331a802..b288508 100644
>> --- a/tests/volume.rc
>> +++ b/tests/volume.rc
>> @@ -579,7 +579,9 @@ function num_graphs
>>  function get_aux()
>>  {
>>  ##Check if a auxiliary mount is there
>> -df -h 2>&1 | sed 's#/build/install##' | grep -e
>> "[[:space:]]/run/gluster/${V0}$" -e "[[:space:]]/var/run/gluster/${V0}$" 
>> -
>> +local rundir=$(gluster --print-statedumpdir)
>> +local pid=$(cat ${rundir}/${V0}.pid)
>> +pidof glusterfs 2>&1 | grep -w $pid
>>
>>  if [ $? -eq 0 ]
>>  then
>>
>
> Based on what I saw in code, this seems to get the job done. Comments
> welcome:
> http://review.gluster.org/14988
>
>
 Nice work Pranith :)
 All, once this is backported to release-3.7, any patches on release-3.7
 patches will need to be rebased so they will pass the NetBSD regression.

>>>
>>> I am suddenly confused about this - will the patches need to be rebased
>>> or with the next run automatically include the changes once Pranith's fix
>>> is merged?
>>>
>>
>> May be someone more knowledgeable about this should confirm this, but at
>> least from the build-log, I don't see any rebase command being executed
>> with origin/master:
>>
>> *04:07:36* Triggered by Gerrit: http://review.gluster.org/13762*04:07:36* 
>> Building remotely on slave26.cloud.gluster.org 
>>  (smoke_tests 
>> rackspace_regression_2gb glusterfs-devrpms) in workspace 
>> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered*04:07:36*  > 
>> git rev-parse --is-inside-work-tree # timeout=10*04:07:36* Fetching changes 
>> from the remote Git repository*04:07:36*  > git config remote.origin.url 
>> git://review.gluster.org/glusterfs.git # timeout=10*04:07:36* Fetching 
>> upstream changes from git://review.gluster.org/glusterfs.git*04:07:36*  > 
>> git --version # timeout=10*04:07:36*  > git -c core.askpass=true fetch 
>> --tags --progress git://review.gluster.org/glusterfs.git 
>> refs/changes/62/13762/4*04:07:44*  > git rev-parse 
>> 838b5c34127edd0450b0449e38f075f56056f2c7^{commit} # timeout=10*04:07:44* 
>> Checking out Revision 838b5c34127edd0450b0449e38f075f56056f2c7 
>> (master)*04:07:44*  > git config core.sparsecheckout # timeout=10*04:07:44*  
>> > git checkout -f 838b5c34127edd0450b0449e38f075f56056f2c7*04:07:45*  > git 
>> rev-parse FETCH_HEAD^{commit} # timeout=10*04:07:45*  > git rev-list 
>> 8cbee639520bf4631ce658e2da9b4bc3010d2eaa # timeout=10*04:07:45*  > git tag 
>> -a -f -m Jenkins Build #22315 
>> jenkins-rackspace-regression-2GB-triggered-22315 # timeout=10
>>
>>
>>
>>
>>>
>>
>> On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran <
>> nbala...@redhat.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
>>> pkara...@redhat.com> wrote:
>>>


 On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran <
 nbala...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy 
> wrote:
>
>> > I attempted to get us more space on NetBSD by creating a new
>> partition called
>> > /data and putting /build as a symlink to /data/build. This has
>> caused
>> > problems
>> > with tests/basic/quota.t. It's marked as bad for master, but
>> not for
>> > release-3.7. This is possibly because we have a hard-coded grep
>> for
>> > /build/install against df -h.
>>
>> For the benefit of anyone else looking at this, the grep actually
>> seems to be
>> in volume.rc and not in the test itself.
>>
>
> That's right -  it appears to have been done to exclude the
> install path components from the df output which is what is being 
> done to
> find the aux mount. Is there a better way to figure out if the aux 
> mount is
> running?
>
>>

Re: [Gluster-devel] readdir() harmful in threaded code

2016-07-23 Thread Pranith Kumar Karampuri

Emmanuel,
   I procrastinated too long on this :-/, It is July already :-(. I
just looked at the man page in Linux and it is a bit confusing, so I am not
sure how to go ahead.

For readdir_r(), I see:

DESCRIPTION
   This function is deprecated; use readdir(3) instead.

   The readdir_r() function was invented as a reentrant version of
readdir(3).  It reads
   the next directory entry from the directory stream dirp, and returns
it in the  call‐
   er-allocated  buffer  pointed  to by entry.  For details of the
dirent structure, see
   readir(3).

For readdir(3) I see:
ATTRIBUTES
   For an explanation of the terms used in this section, see
attributes(7).

   ┌──┬───┬──┐
   │Interface │ Attribute │ Value│
   ├──┼───┼──┤
   │readdir() │ Thread safety │ MT-Unsafe race:dirstream │
   └──┴───┴──┘

   In the current POSIX.1 specification (POSIX.1-2008), readdir() is
not required to be thread-safe.  However, in modern implementations
(including the glibc implementation), concur‐
   rent  calls  to readdir() that specify different directory streams
are thread-safe.  In cases where multiple threads must read from the same
directory stream, using readdir() with
   external synchronization is still preferable to the use of the
deprecated readdir_r(3) function.  It is expected that a future version of
POSIX.1 will require  that  readdir()  be
   thread-safe when concurrently employed on different directory
streams.

So should we do readdir() with external locks for everything instead?

On Thu, Feb 11, 2016 at 2:35 PM, Emmanuel Dreyfus  wrote:

> Juste to make sure there is no misunderstanding here: unfortunately I
> do not have time right now to submit a fix. It would be nice if someone
> else coule look at it.
>
> On Wed, Feb 10, 2016 at 01:48:52PM +, Emmanuel Dreyfus wrote:
> > Hi
> >
> > After obtaining a core in a regression, I noticed there are a few
> readdir()
> > use in threaded code. This is begging for a crash, as readdir() maintains
> > an internal state that will be trashed on concurent use. readdir_r()
> > should be used instead.
> >
> > A quick search shows readdir(à usage here:
> > contrib/fuse-util/mount_util.c:30
> > extras/test/ld-preload-test/ld-preload-test.c:310
> > extras/test/test-ffop.c:550
> > libglusterfs/src/compat.c:256
> > libglusterfs/src/compat.c:315
> > libglusterfs/src/syscall.c:97
> > tests/basic/fops-sanity.c:662
> > tests/utils/arequal-checksum.c:331
> >
> > Occurences in contrib, extra and tests are probably harmless are there
> > are usage in standalone programs that are not threaded. We are left with
> > three groups of problems:
> >
> > 1) libglusterfs/src/compat.c:256 and libglusterfs/src/compat.c:315
> > This is Solaris compatibility code. Is it used at all?
> >
> > 2)  libglusterfs/src/syscall.c:97 This is the sys_readdir() wrapper,
> > which is in turn used in:
> > libglusterfs/src/run.c:284
> > xlators/features/bit-rot/src/stub/bit-rot-stub-helpers.c:582
> > xlators/features/changelog/lib/src/gf-history-changelog.c:854
> > xlators/features/index/src/index.c:471
> > xlators/mgmt/glusterd/src/glusterd-snapshot-utils.c
> > xlators/storage/posix/src/posix.c:3700
> > xlators/storage/posix/src/posix.c:5896
> >
> > 3) We also find sys_readdir() in libglusterfs/src/common-utils.h for
> > GF_FOR_EACH_ENTRY_IN_DIR() which in turn appears in:
> > libglusterfs/src/common-utils.c:3979
> > libglusterfs/src/common-utils.c:4002
> > xlators/mgmt/glusterd/src/glusterd-hooks.c:365
> > xlators/mgmt/glusterd/src/glusterd-hooks.c:379
> > xlators/mgmt/glusterd/src/glusterd-store.c:651
> > xlators/mgmt/glusterd/src/glusterd-store.c:661
> > xlators/mgmt/glusterd/src/glusterd-store.c:1781
> > xlators/mgmt/glusterd/src/glusterd-store.c:1806
> > xlators/mgmt/glusterd/src/glusterd-store.c:3044
> > xlators/mgmt/glusterd/src/glusterd-store.c:3072
> > xlators/mgmt/glusterd/src/glusterd-store.c:3593
> > xlators/mgmt/glusterd/src/glusterd-store.c:3606
> > xlators/mgmt/glusterd/src/glusterd-store.c:4032
> > xlators/mgmt/glusterd/src/glusterd-store.c:4111
> >
> > There a hive of sprious bugs to squash here.
> >
> > --
> > Emmanuel Dreyfus
> > m...@netbsd.org
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
>
> --
> Emmanuel Dreyfus
> m...@netbsd.org
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>

-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

2016-07-23 Thread Atin Mukherjee

AFAIK, an explicit rebase is required.

On Saturday 23 July 2016, Pranith Kumar Karampuri 
wrote:

>
>
> On Sat, Jul 23, 2016 at 10:17 AM, Nithya Balachandran  > wrote:
>
>>
>>
>> On Sat, Jul 23, 2016 at 9:45 AM, Nithya Balachandran > > wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 9:07 PM, Pranith Kumar Karampuri <
>>> pkara...@redhat.com
>>> > wrote:
>>>


 On Fri, Jul 22, 2016 at 8:12 PM, Pranith Kumar Karampuri <
 pkara...@redhat.com
 > wrote:

> I am playing with the following diff, let me see.
>
> diff --git a/tests/volume.rc b/tests/volume.rc
> index 331a802..b288508 100644
> --- a/tests/volume.rc
> +++ b/tests/volume.rc
> @@ -579,7 +579,9 @@ function num_graphs
>  function get_aux()
>  {
>  ##Check if a auxiliary mount is there
> -df -h 2>&1 | sed 's#/build/install##' | grep -e
> "[[:space:]]/run/gluster/${V0}$" -e "[[:space:]]/var/run/gluster/${V0}$" -
> +local rundir=$(gluster --print-statedumpdir)
> +local pid=$(cat ${rundir}/${V0}.pid)
> +pidof glusterfs 2>&1 | grep -w $pid
>
>  if [ $? -eq 0 ]
>  then
>

 Based on what I saw in code, this seems to get the job done. Comments
 welcome:
 http://review.gluster.org/14988


>>> Nice work Pranith :)
>>> All, once this is backported to release-3.7, any patches on release-3.7
>>> patches will need to be rebased so they will pass the NetBSD regression.
>>>
>>
>> I am suddenly confused about this - will the patches need to be rebased
>> or with the next run automatically include the changes once Pranith's fix
>> is merged?
>>
>
> May be someone more knowledgeable about this should confirm this, but at
> least from the build-log, I don't see any rebase command being executed
> with origin/master:
>
> *04:07:36* Triggered by Gerrit: http://review.gluster.org/13762*04:07:36* 
> Building remotely on slave26.cloud.gluster.org 
>  (smoke_tests 
> rackspace_regression_2gb glusterfs-devrpms) in workspace 
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered*04:07:36*  > 
> git rev-parse --is-inside-work-tree # timeout=10*04:07:36* Fetching changes 
> from the remote Git repository*04:07:36*  > git config remote.origin.url 
> git://review.gluster.org/glusterfs.git # timeout=10*04:07:36* Fetching 
> upstream changes from git://review.gluster.org/glusterfs.git*04:07:36*  > git 
> --version # timeout=10*04:07:36*  > git -c core.askpass=true fetch --tags 
> --progress git://review.gluster.org/glusterfs.git 
> refs/changes/62/13762/4*04:07:44*  > git rev-parse 
> 838b5c34127edd0450b0449e38f075f56056f2c7^{commit} # timeout=10*04:07:44* 
> Checking out Revision 838b5c34127edd0450b0449e38f075f56056f2c7 
> (master)*04:07:44*  > git config core.sparsecheckout # timeout=10*04:07:44*  
> > git checkout -f 838b5c34127edd0450b0449e38f075f56056f2c7*04:07:45*  > git 
> rev-parse FETCH_HEAD^{commit} # timeout=10*04:07:45*  > git rev-list 
> 8cbee639520bf4631ce658e2da9b4bc3010d2eaa # timeout=10*04:07:45*  > git tag -a 
> -f -m Jenkins Build #22315 jenkins-rackspace-regression-2GB-triggered-22315 # 
> timeout=10
>
>
>
>
>>
>
> On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran <
> nbala...@redhat.com
> > wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
>> pkara...@redhat.com
>> > wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran <
>>> nbala...@redhat.com
>>> > wrote:
>>>


 On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy > wrote:

> > I attempted to get us more space on NetBSD by creating a new
> partition called
> > /data and putting /build as a symlink to /data/build. This has
> caused
> > problems
> > with tests/basic/quota.t. It's marked as bad for master, but not
> for
> > release-3.7. This is possibly because we have a hard-coded grep
> for
> > /build/install against df -h.
>
> For the benefit of anyone else looking at this, the grep actually
> seems to be
> in volume.rc and not in the test itself.
>

 That's right -  it appears to have been done to exclude the install
 path components from the df output which is what is being done to find 
 the
 aux mount. Is there a

Re: [Gluster-devel] 3.7 regressions on NetBSD

2016-07-23 Thread Pranith Kumar Karampuri

On Sat, Jul 23, 2016 at 10:17 AM, Nithya Balachandran 
wrote:

>
>
> On Sat, Jul 23, 2016 at 9:45 AM, Nithya Balachandran 
> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 9:07 PM, Pranith Kumar Karampuri <
>> pkara...@redhat.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 8:12 PM, Pranith Kumar Karampuri <
>>> pkara...@redhat.com> wrote:
>>>
 I am playing with the following diff, let me see.

 diff --git a/tests/volume.rc b/tests/volume.rc
 index 331a802..b288508 100644
 --- a/tests/volume.rc
 +++ b/tests/volume.rc
 @@ -579,7 +579,9 @@ function num_graphs
  function get_aux()
  {
  ##Check if a auxiliary mount is there
 -df -h 2>&1 | sed 's#/build/install##' | grep -e
 "[[:space:]]/run/gluster/${V0}$" -e "[[:space:]]/var/run/gluster/${V0}$" -
 +local rundir=$(gluster --print-statedumpdir)
 +local pid=$(cat ${rundir}/${V0}.pid)
 +pidof glusterfs 2>&1 | grep -w $pid

  if [ $? -eq 0 ]
  then

>>>
>>> Based on what I saw in code, this seems to get the job done. Comments
>>> welcome:
>>> http://review.gluster.org/14988
>>>
>>>
>> Nice work Pranith :)
>> All, once this is backported to release-3.7, any patches on release-3.7
>> patches will need to be rebased so they will pass the NetBSD regression.
>>
>
> I am suddenly confused about this - will the patches need to be rebased or
> with the next run automatically include the changes once Pranith's fix is
> merged?
>

May be someone more knowledgeable about this should confirm this, but at
least from the build-log, I don't see any rebase command being executed
with origin/master:

*04:07:36* Triggered by Gerrit:
http://review.gluster.org/13762*04:07:36* Building remotely on
slave26.cloud.gluster.org

(smoke_tests rackspace_regression_2gb glusterfs-devrpms) in workspace
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered*04:07:36*
 > git rev-parse --is-inside-work-tree # timeout=10*04:07:36* Fetching
changes from the remote Git repository*04:07:36*  > git config
remote.origin.url git://review.gluster.org/glusterfs.git #
timeout=10*04:07:36* Fetching upstream changes from
git://review.gluster.org/glusterfs.git*04:07:36*  > git --version #
timeout=10*04:07:36*  > git -c core.askpass=true fetch --tags
--progress git://review.gluster.org/glusterfs.git
refs/changes/62/13762/4*04:07:44*  > git rev-parse
838b5c34127edd0450b0449e38f075f56056f2c7^{commit} #
timeout=10*04:07:44* Checking out Revision
838b5c34127edd0450b0449e38f075f56056f2c7 (master)*04:07:44*  > git
config core.sparsecheckout # timeout=10*04:07:44*  > git checkout -f
838b5c34127edd0450b0449e38f075f56056f2c7*04:07:45*  > git rev-parse
FETCH_HEAD^{commit} # timeout=10*04:07:45*  > git rev-list
8cbee639520bf4631ce658e2da9b4bc3010d2eaa # timeout=10*04:07:45*  > git
tag -a -f -m Jenkins Build #22315
jenkins-rackspace-regression-2GB-triggered-22315 # timeout=10




>

 On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran <
 nbala...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran <
>> nbala...@redhat.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy 
>>> wrote:
>>>
 > I attempted to get us more space on NetBSD by creating a new
 partition called
 > /data and putting /build as a symlink to /data/build. This has
 caused
 > problems
 > with tests/basic/quota.t. It's marked as bad for master, but not
 for
 > release-3.7. This is possibly because we have a hard-coded grep
 for
 > /build/install against df -h.

 For the benefit of anyone else looking at this, the grep actually
 seems to be
 in volume.rc and not in the test itself.

>>>
>>> That's right -  it appears to have been done to exclude the install
>>> path components from the df output which is what is being done to find 
>>> the
>>> aux mount. Is there a better way to figure out if the aux mount is 
>>> running?
>>>

 > Nithya has spent the last 2 days debugging
 > without much success. What's a good way forward here? Mark the
 test as
 > failing for 3.7?

>>>
>>> Right. Something went wrong with the system and it refused to run
>>> the tests after a while.
>>>
>>>

 I don't think so.  There are 13 tests that use the affected function
 (get_aux).  Do we want to disable 13 tests?  I think we actually
 need
 to fix the function instead.  It seems to me that the check we're
 making is very hacky in two ways:

Checking for both /run and /var/run instead

Re: [Gluster-devel] readdir() harmful in threaded code

Re: [Gluster-devel] 3.7 regressions on NetBSD

Re: [Gluster-devel] readdir() harmful in threaded code

Re: [Gluster-devel] 3.7 regressions on NetBSD

Re: [Gluster-devel] 3.7 regressions on NetBSD

Re: [Gluster-devel] readdir() harmful in threaded code

Re: [Gluster-devel] 3.7 regressions on NetBSD

Re: [Gluster-devel] 3.7 regressions on NetBSD

8 matches

Site Navigation

Mail list logo

Footer information