Re: [Gluster-devel] Need help with running bench test on Gluster cluster.

[Menaka Mohan]

Hello Nigel B,

Thank you so much for you reply. Recently, I have noticed few syntax errors on 
the code and I have corrected them. Kindly review the Pull request.

Since I do not have a laptop with good specifications, I am unable to use VMs. 
I tried and it crashed many times. So with the support of a college friend I 
have setup the environment in a lab and accessing it through Bitvise SSH 
client. With the process becoming more complex, I am learning more things.

I have thoroughly gone through Shyam's reply and working on it.

Regards,
Menaka M


On Tue, Oct 11, 2016 at 07:53:09PM +, Menaka Mohan wrote:
> Hi,
>
>
> I am Menaka M. I am interested in participating in this round of Outreachy 
> under Gluster. I am new to this open source world. Kindly help me with the 
> following query.
>
>
> I have set up the Gluster development environment with two servers and one 
> client. I am trying to run the basic bench test on the Gluster cluster from 
> this GitHub 
> repo. 
> I also have IOZone installed. While trying to run the provided script, i get 
> the following error. I was trying to identify the cause of the error. Kindly 
> help me with that.
>
> So, I learned more about IOZone and also the performance testing section in 
> the Gluster docs. With that knowledge and to learn more, I have gone through 
> the code and running iozone commands mentioned in the GlusterBench.py script 
> individually.
>
> If I am asking a very basic thing, apologies. I will quickly learn things.
>
> Regards,
> Menaka M
>

Hi Menaka,

Thanks for taking a look at gbench. It appears that you have uncovered a bug in 
our documentation and code.

Shyam is the expert here and I'll let him chime in. You need an environment 
variable with CLIENTS and SERVER to get this working. So when you run this, you 
need at least two machines (VMs would be best. Happy to guide you on how to set 
this up if you'd like). It would look something like this:

CLIENTS="192.168.0.1 192.158.0.2" SERVERS="192.168.0.3 192.168.0.3" python 
GlusterBench.py -s 64 -f 1 -n 5 -v

I'm making a lot of assumptions here. If there's anything that is confusing, 
let me know and I'm happy to clarify. I'll let Shyam jump in later today (he's 
in EST) to chime with anything else I've missed.

--
nigelb
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] quota-rename.t core in netbsd

[Raghavendra Gowdappa]

+Muthu, +Sanoj

Sure Vijay. One of us will take a look.

- Original Message -
> From: "Vijay Bellur" 
> To: "Gluster Devel" , "Raghavendra Gowdappa" 
> 
> Sent: Wednesday, October 5, 2016 11:25:34 PM
> Subject: quota-rename.t core in netbsd
> 
> Hi All,
> 
> I observed a few crashes due to quota-rename.t in netbsd regression
> runs [1] [2].  Raghavendra - can you please take a look when you get a
> chance?
> 
> The core files and logs cannot be downloaded from the URLs in jenkins
> job console history for NetBSD. I have logged a bug [3] on the
> infrastructure for that.
> 
> Thanks,
> Vijay
> 
> [1] https://build.gluster.org/job/netbsd7-regression/942/consoleFull
> 
> [2]  https://build.gluster.org/job/netbsd7-regression/945/console
> 
> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1382097
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[sriram]

Hi Avra,

Could you let me know on the below request?

Sriram


On Tue, Oct 4, 2016, at 11:16 AM, sri...@marirs.net.in wrote:
> Hi Avra,
>
> I checked the comment, the series of patches, (There are nine patches)
> for which I've posted for a review below. They've all the necessary
> makefiles to compile.
>
> Would you want me to consolidate all'em and post them as a single
> patch? (I thought that would be a little confusing, since it'd changes
> with different intentions).
>
> Sriram
>
>
> On Mon, Oct 3, 2016, at 03:54 PM, Avra Sengupta wrote:
>> Hi Sriram,
>>
>> I posted a comment into the first patch. It doesn't compile by
>> itself. We need to update the respective makefiles to be able to
>> compile it. Then we can introduce the tabular structure in the same
>> patch to have the framework set for the zfs snapshots. Thanks.
>>
>> Regards,
>> Avra
>>
>> On 09/30/2016 10:24 AM, sri...@marirs.net.in wrote:
>>> Hi Avra,
>>>
>>> Could you have a look into the below request?
>>>
>>> Sriram
>>>
>>>
>>> On Fri, Sep 23, 2016, at 04:10 PM, sri...@marirs.net.in wrote:
 Hi Avra,

 Have submitted the patches for Modularizing snapshot,

 https://bugzilla.redhat.com/show_bug.cgi?id=1377437

 This is the patch set:

  http://review.gluster.org/15554 This patch follows the discussion
  from the gluster-devel mail chain of, ...
  http://review.gluster.org/1 Referring to bugID:1377437,
  Modularizing snapshot for plugin based modules.
  http://review.gluster.org/15556 - This is third patch in the
  series for the bug=1377437
  http://review.gluster.org/15557 [BugId:1377437][Patch4]: Refering
  to the bug ID,
  http://review.gluster.org/15558 [BugId:1377437][Patch5]: Refering
  to the bug ID,
  http://review.gluster.org/15559 [BugId:1377437][Patch6]: Refering
  to the bug ID,
  http://review.gluster.org/15560 [BugId:1377437][Patch7]: Refering
  to the bug ID. * This patch has some minor ...
  http://review.gluster.org/15561 [BugId:1377437][Patch8]: Refering
  to the bug ID, this commit has minor fixes ...
  http://review.gluster.org/15562 [BugId:1377437][Patch9]: Refering
  to the bug ID, - Minor header file ...

 Primarily, focused on moving lvm based implementation into plugins.
 Have spread the commits across nine patches, some of them are
 minors, except a couple of ones which does the real work. Others
 are minors. Followed this method since, it would be easy for a
 review (accept/reject). Let me know if there is something off the
 methods followed with gluster devel. Thanks

 Sriram

 On Mon, Sep 19, 2016, at 10:58 PM, Avra Sengupta wrote:
> Hi Sriram,
>
> I have created a bug for this
> (https://bugzilla.redhat.com/show_bug.cgi?id=1377437). The plan is
> that for the first patch as mentioned below, let's not meddle with
> the zfs code at all. What we are looking at is segregating the lvm
> based code as is today, from the management infrastructure (which
> is addressed in your patch), and creating a table based pluggable
> infra(refer to gd_svc_cli_actors[] in xlators/mgmt/glusterd/src/glusterd-
> handler.c and other similar tables in gluster code base to get a
> understanding of what I am conveying), which can be used to call
> this code and still achieve the same results as we do today.
>
> Once this code is merged, we can use the same infra to start
> pushing in the zfs code (rest of your current patch). Please let
> me know if you have further queries regarding this. Thanks.
>
> Regards,
> Avra
>
> On 09/19/2016 07:52 PM, sri...@marirs.net.in wrote:
>> Hi Avra,
>>
>> Do you have a bug id for this changes? Or may I raise a new one?
>>
>> Sriram
>>
>>
>> On Fri, Sep 16, 2016, at 11:37 AM, sri...@marirs.net.in wrote:
>>> Thanks Avra,
>>>
>>> I'll send this patch to gluster master in a while.
>>>
>>> Sriram
>>>
>>>
>>> On Wed, Sep 14, 2016, at 03:08 PM, Avra Sengupta wrote:
 Hi Sriram,

 Sorry for the delay in response. I started going through the
 commits in the github repo. I finished going through the first
 commit, where you create a plugin structure and move code.
 Following is the commit link:

 https://github.com/sriramster/glusterfs/commit/7bf157525539541ebf0aa36a380bbedb2cae5440

 FIrst of all, the overall approach of using plugins, and
 maintaining plugins that is used in the patch is in sync with
 what we had discussed. There are some gaps though, like in the
 zfs functions the snap brick is mounted without updating
 labels, and in restore you perform a zfs rollback, which
 significantly changes the behavior between how a lvm based
 snapshot and a zfs based snapshot.

 

Re: [Gluster-devel] [Outreachy] Need help with running bench test on Gluster cluster.

[Shyam]

Took the liberty of adding [Outreachy] to the subject, makes it easier 
for others to search as well (probably). If people have objections, let 
me know and I will post a single response here removing the same for the 
future.




On 10/11/2016 03:53 PM, Menaka Mohan wrote:

Hi,


I am Menaka M. I am interested in participating in this round of
Outreachy under Gluster. I am new to this open source world. Kindly help
me with the following query.


Welcome!




I have set up the Gluster development environment with two servers and
one client. I am trying to run the basic bench test on the Gluster
cluster from this GitHub repo
. I
also have IOZone installed. While trying to run the provided script, i
get the following error. I was trying to identify the cause of the
error. Kindly help me with that.


It looks like you have made progress (from your IRC ping on clients.ioz 
and its contents). I guess you got the CLIENTS, SERVERS and some other 
prerequisites right. It also looks like you either figured out how to 
setup rsh for iozone, or exported RSH env-var as 'ssh'.  So some good 
progress there.


It would be nice if you could take some time out to add to the README 
these additional prerequisites. (when you get the time) (you would be 
looking at github pull requests [1] for the same)




So, I learned more about IOZone and also the performance testing section
in the Gluster docs. With that knowledge and to learn more, I have gone
through the code and running iozone commands mentioned in the
GlusterBench.py script individually.

If I am asking a very basic thing, apologies. I will quickly learn things.


Nope! not necessarily basic, as I got some failures too when using 
latest iozone binaries/sources. One such failure was in [2] where the 
latest iozone results are changed to report "kB/...", rather than 
"KB/..." and so the parsing failed. Anyway, I made a local fix to my 
python script for the same (will push the change out soonish).


So coming to your test failure:

- Could you post your full log using something like fpaste, and share 
the link (helps not making any assumptions on my part) [3]


- I hit a similar failure in my tests, see [4]. This run seems to have 
hit some ssh connection issue, because of which a sample was left as 
"None" (which is the default)


- In your case *all* samples are left as "None". So I suspect a more 
generic parsing failure of the results as obtained from *every* iozone run


- To detect what failed and why, it would be better to take a look at 
the entire log


- Additionally, you could also add a few prints within the 
extract_iozone_result function to debug the root cause of failure




Regards,
Menaka M

--

python GlusterBench.py -s 64 -f 1 -n 5 -v

Number threads = 4
Client list = HadoopSlave4

Running IOZone with 64KB record size and 4 threads,  Creating an 8 GB
file with every thread.
Running smallfile with 64KB files, creating 1 files.
Running squential IOZone tests, starting with sequential writes.

About to gather sample --> 0

Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
 Al Slater, Scott Rhine, Mike Wisner, Ken Goss
 Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
 Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
 Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,


Were there any additional output lines here? I do understand that you 
have kept the output terse later, but to understand parsing errors, the 
output from iozone would be helpful.




Adding the current sample to the list: None


The above line is where the parsing has failed, it should retrieve a 
number and not a default string "None" from the routine.



Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution


I think the default shell in your environment is not bash, and that 
leads to the above "Bad substitution". On a hunch I am providing a stack 
overflow link for the same [5]. This should not have a repercussion on 
the test result, but I could be mistaken.



Cleaning up files.

About to gather sample --> 1

/* I removed the Iozone and Contributors header */

Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
Cleaning up files.

About to gather sample --> 2

Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
Cleaning up files.

About to gather sample --> 3

Adding the current sample to the list: None
Dropping cache

Re: [Gluster-devel] Spurious failure of ./tests/bugs/glusterd/bug-913555.t

[Atin Mukherjee]

So the test fails (intermittently) in check_fs which tries to do a df on
the mount point for a volume which is carved out of three bricks from 3
nodes and one node is completely down. A quick look at the mount log
reveals the following:

[2016-10-10 13:58:59.279446]:++
G_LOG:./tests/bugs/glusterd/bug-913555.t:
TEST: 48 0 check_fs /mnt/glusterfs/0 ++
[2016-10-10 13:58:59.287973] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk]
0-patchy-client-2:  remote operation failed. Path: /
(----0001) [Transport endpoint is not connected]
[2016-10-10 13:58:59.288326] I [MSGID: 109063]
[dht-layout.c:713:dht_layout_normalize]
0-patchy-dht: Found anomalies in / (gfid =
----0001).
Holes=1 overlaps=0
[2016-10-10 13:58:59.288352] W [MSGID: 109005]
[dht-selfheal.c:2102:dht_selfheal_directory]
0-patchy-dht: Directory  selfheal failed: 1 subvolumes down.Not fixing.
path = /, gfid =
[2016-10-10 13:58:59.288643] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk]
0-patchy-client-2:  remote operation failed. Path: /
(----0001) [Transport endpoint is not connected]
[2016-10-10 13:58:59.288927] W [fuse-resolve.c:132:fuse_resolve_gfid_cbk]
0-fuse: ----   0001: failed to resolve
(Stale file handle)
[2016-10-10 13:58:59.288949] W [fuse-bridge.c:2597:fuse_opendir_resume]
0-glusterfs-fuse: 7: OPENDIR (-- --0001)
resolution failed
[2016-10-10 13:58:59.289505] W [fuse-resolve.c:132:fuse_resolve_gfid_cbk]
0-fuse: ----   0001: failed to resolve
(Stale file handle)
[2016-10-10 13:58:59.289524] W [fuse-bridge.c:3137:fuse_statfs_resume]
0-glusterfs-fuse: 8: STATFS (--   --0001)
resolution fail

DHT  team - are these anomalies expected here? I also see opendir and
statfs failing here too.

On Wed, Oct 12, 2016 at 12:18 PM, Atin Mukherjee 
wrote:

> I will take a look at it in sometime.
>
> On Wed, Oct 12, 2016 at 12:08 PM, Oleksandr Natalenko <
> oleksa...@natalenko.name> wrote:
>
>> Hello.
>>
>> Vijay asked me to drop a note about spurious failure of
>> ./tests/bugs/glusterd/bug-913555.t test. Here are the examples:
>>
>> * https://build.gluster.org/job/centos6-regression/1069/consoleFull
>> * https://build.gluster.org/job/centos6-regression/1076/consoleFull
>>
>> Could someone take a look at it?
>>
>> Also, last two tests were broken because of this:
>>
>> ===
>> Slave went offline during the build
>> ===
>>
>> See these builds for details:
>>
>> * https://build.gluster.org/job/centos6-regression/1077/consoleFull
>> * https://build.gluster.org/job/centos6-regression/1078/consoleFull
>>
>> Was that intentionally?
>>
>> Thanks.
>>
>> Regards,
>>   Oleksandr
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
>
> --Atin
>



-- 

--Atin
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Interested in contributing to Gluster Documentation

[Humble Devassy Chirammal]

Hi Srravya,

Welcome to Gluster Community !

>
I am Srravya. I recently came across Gluster on the list of
communities/organisations participating in Outreachy
. Upon going through
the list of projects Gluster has listed for the internship, I noticed that
one of them is a documentation project. As a new comer to Gluster, I
believe this would be a good way for me to get started in understanding
Gluster and make my first contributions.
>

Yes, it will give a good start and introduction to the  community.

>
I would love to interact with community members including developers and
documentation team members to know how best I can begin working on Gluster.
It would be great if I could be pointed to any documentation bugs/tasks I
could pick and also general docs on getting started and initial set up
steps.
>

Give a start with the workflow which Nigel mentioned.

Please feel free to open issues/PRs in github documentation project and if
need help, ping us on #gluster or #gluster-dev.


--Humble
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Interested in contributing to Gluster Documentation

[Nigel Babu]

On Tue, Oct 04, 2016 at 02:22:35PM +0530, Srravya C wrote:
> Hi all,
>
> I am Srravya. I recently came across Gluster on the list of
> communities/organisations participating in Outreachy
> . Upon going through
> the list of projects Gluster has listed for the internship, I noticed that
> one of them is a documentation project. As a new comer to Gluster, I
> believe this would be a good way for me to get started in understanding
> Gluster and make my first contributions.
>
> I would love to interact with community members including developers and
> documentation team members to know how best I can begin working on Gluster.
> It would be great if I could be pointed to any documentation bugs/tasks I
> could pick and also general docs on getting started and initial set up
> steps.
>
> Looking forward to collaborating :)
>
> Cheers,
> Srravya

Hi Srravya,

I suspect you pinged me on IRC over the last week trying to find help.
Apologies in the delay in getting back to you. We've all been at Gluster
Developer Summit and was mostly away from our computers.

I'm speaking on behalf of Amye and documentation team members here, so you can
get started right way. A good thing to try is the following:

1. Clone the documentation repo[1].
2. Try to build the documentation (Documentation for mkdocs[2]).

The initial contribution we've listed is for you to find something that needs
improvement and fix it. I would recommend taking a look around to see if you
can find something to do. The bugs we have on file aren't very useful for
initial contributions.

I have a couple of suggestions on something you can start with
1. Install aspell[3] and check for spelling mistakes across the documentation.
   Submit pull requests for fxing them.
2. Install Linkchecker[4] and run it against gluster.readthedocs.org and catch
   any links that lead to 404.

If you have questions, please free to ask on the list and we're happy to help.

[1] https://github.com/gluster/glusterdocs/
[2] http://www.mkdocs.org/
[3] http://aspell.net/
[4] https://pypi.python.org/pypi/LinkChecker

--
nigelb
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Need help with running bench test on Gluster cluster.

[Nigel Babu]

On Tue, Oct 11, 2016 at 07:53:09PM +, Menaka Mohan wrote:
> Hi,
>
>
> I am Menaka M. I am interested in participating in this round of Outreachy 
> under Gluster. I am new to this open source world. Kindly help me with the 
> following query.
>
>
> I have set up the Gluster development environment with two servers and one 
> client. I am trying to run the basic bench test on the Gluster cluster from 
> this GitHub 
> repo. 
> I also have IOZone installed. While trying to run the provided script, i get 
> the following error. I was trying to identify the cause of the error. Kindly 
> help me with that.
>
> So, I learned more about IOZone and also the performance testing section in 
> the Gluster docs. With that knowledge and to learn more, I have gone through 
> the code and running iozone commands mentioned in the GlusterBench.py script 
> individually.
>
> If I am asking a very basic thing, apologies. I will quickly learn things.
>
> Regards,
> Menaka M
>

Hi Menaka,

Thanks for taking a look at gbench. It appears that you have uncovered a bug in
our documentation and code.

Shyam is the expert here and I'll let him chime in. You need an environment
variable with CLIENTS and SERVER to get this working. So when you run this, you
need at least two machines (VMs would be best. Happy to guide you on how to set
this up if you'd like). It would look something like this:

CLIENTS="192.168.0.1 192.158.0.2" SERVERS="192.168.0.3 192.168.0.3" python 
GlusterBench.py -s 64 -f 1 -n 5 -v

I'm making a lot of assumptions here. If there's anything that is confusing,
let me know and I'm happy to clarify. I'll let Shyam jump in later today (he's
in EST) to chime with anything else I've missed.

--
nigelb
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Why a -2 need to be carried over to next patch set?

[Niels de Vos]

On Wed, Oct 12, 2016 at 11:09:37AM +0530, Atin Mukherjee wrote:
> Although a -2 on a patch indicates that reviewer has strongly disagreed on
> the changes done on the patch, but is it right to carry forward the same
> vote on the subsequent patch set(s)? What if the changes on the following
> patch sets are in line with the comments on the patch set where a -2 was
> mentioned? As it stands until the same reviewer revokes the -2, the patch
> can't be merged. Is this what was intended for?

Yes, otherwise the reviewer should have used -1.

> My primary concern here is if the concerned person is unavailable (for
> various reason) the acceptance of the patch gets delayed even if we have co
> maintainers for the same module acking the patch?

If people give -2 for components they are not responsoble for, I think
we should educate them. Maintainers of components or stable versions
should be allowed to give -2 (or +2), others can give that as well, but
it should be seen as a -1 (or +1) in that case.

For patches that are really urgent, and the -2 reviewer does not respond
in a few days/week, the Gerrit admins can remove the -2. For these cases
I would expect an email to the reviewer, with gluster-devel on CC.

> What do others think here? Should we continue to carry over a -2 on the
> subsequent patch sets?

Yes.

Niels


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Spurious failure of ./tests/bugs/glusterd/bug-913555.t

[Atin Mukherjee]

I will take a look at it in sometime.

On Wed, Oct 12, 2016 at 12:08 PM, Oleksandr Natalenko <
oleksa...@natalenko.name> wrote:

> Hello.
>
> Vijay asked me to drop a note about spurious failure of
> ./tests/bugs/glusterd/bug-913555.t test. Here are the examples:
>
> * https://build.gluster.org/job/centos6-regression/1069/consoleFull
> * https://build.gluster.org/job/centos6-regression/1076/consoleFull
>
> Could someone take a look at it?
>
> Also, last two tests were broken because of this:
>
> ===
> Slave went offline during the build
> ===
>
> See these builds for details:
>
> * https://build.gluster.org/job/centos6-regression/1077/consoleFull
> * https://build.gluster.org/job/centos6-regression/1078/consoleFull
>
> Was that intentionally?
>
> Thanks.
>
> Regards,
>   Oleksandr
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>



-- 

--Atin
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Spurious failure of ./tests/bugs/glusterd/bug-913555.t

[Oleksandr Natalenko]

Hello.

Vijay asked me to drop a note about spurious failure of 
./tests/bugs/glusterd/bug-913555.t test. Here are the examples:


* https://build.gluster.org/job/centos6-regression/1069/consoleFull
* https://build.gluster.org/job/centos6-regression/1076/consoleFull

Could someone take a look at it?

Also, last two tests were broken because of this:

===
Slave went offline during the build
===

See these builds for details:

* https://build.gluster.org/job/centos6-regression/1077/consoleFull
* https://build.gluster.org/job/centos6-regression/1078/consoleFull

Was that intentionally?

Thanks.

Regards,
  Oleksandr
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel