[Gluster-devel] [OUTREACHY] Application

[Sara Silva]

Hi!
My name is Sara Silva. I am a candidate for the 17th round of Outreach.
I am a student in the fifth semester in Software Engineering in Brazil, and
I have a reasonable experience of C, ShelI and Python, mainly I used them
in university activities. In addition to being very interested in
distributed systems
I want to subscribe to the Gluster project to extend libgfapi
Already following the first instructions in "Tasks during the enrollment
period" (
https://www.outreachy.org/december-2018-march-2019-outreachy-internships/communities/gluster/
).
Thankful.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] OUTREACHY (Extend Python and Go bindings for libgfapi library in GlusterFS)

[Prashanth Pai]

Hi Anshul,

You'll find all the details on the project page:
https://www.outreachy.org/communities/cfp/gluster/project/extend-python-and-go-bindings-for-libgfapi-library/

You'll have to do application period tasks prior to taking up issues from
the project.
Once you are done with the application period tasks, please send the
details (screenshots, programs) to one or many co-mentors.

On Fri, Oct 12, 2018 at 5:35 PM Anshul tale  wrote:

> Hi,
>
> I would like to work for the project "Extend Python and Go bindings for
> libgfapi library in GlusterFS. Can you please give me some issue to work on
> ?
>
> Thanks
> Anshul Tale
>
>
> "
>


-- 
- Prashanth Pai
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] OUTREACHY (Extend Python and Go bindings for libgfapi library in GlusterFS)

[Anshul tale]

Hi,

I would like to work for the project "Extend Python and Go bindings for
libgfapi library in GlusterFS. Can you please give me some issue to work on
?

Thanks
Anshul Tale


"
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [OUTREACHY]

[Amye Scavarda]

Hi there!
All of our details are listed at:
https://www.outreachy.org/december-2018-march-2019-outreachy-internships/communities/gluster/#extend-python-and-go-bindings-for-libgfapi-library

We suggest installing Gluster as a starting point.
- amye

On Sun, Sep 23, 2018 at 2:18 AM pooja agarwal  wrote:

> Hello I am Pooja Agarwal an undergraduate student from College Of
> Engineering and Technology, Bhubaneswar. I landed on to this project
> through outreachy and would like to contribute to this project. Please
> guide me with the baby steps to start working on this project.
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel



-- 
Amye Scavarda | a...@redhat.com | Gluster Community Lead
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] [OUTREACHY]

[pooja agarwal]

Hello I am Pooja Agarwal an undergraduate student from College Of
Engineering and Technology, Bhubaneswar. I landed on to this project
through outreachy and would like to contribute to this project. Please
guide me with the baby steps to start working on this project.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Outreachy intern's update.

[Menaka Mohan]

Hello all,


I am Menaka M from India, an Outreachy intern working under Gluster. My tasks 
for this internship is to work on the parts of this feature page - Initial 
feature description for improved io-stats [1].


I am working under the mentorship of Shyam and the program co-ordinator Amye.


So far, I have spent time on understanding the Gluster architecture and also 
about the io-stats xlator. I have also contributed few changes to the same 
under the mentor's guidance.


Initially, I have found the missing FOPs in it and then added them to it as my 
initial commit. The commit has been reviewed and merged. [2]


For adding the missing FOPs, I added the compound FOP to the list of FOPs in 
the generator.py file but later understood the problems behind that. The same 
was also explained in the commit by the community members. [3]


And then, I got a task assigned to add the code generation framework for the 
io-stats xlator. The review of the same commit is pending. [4]


My next immediate task is to do the following changes,


For now, for each FOP we get Min/Avg/Max values on profiling. I shall be 
working to also display a histogram based spread as it is just few OPs that is 
taking higher time skewing the average, etc., I shall work on that and other 
parts that are mentioned in the feature page and post regular updates in this 
mailing list.


[1] 
https://review.gluster.org/#/c/16558/
 , Feature page

[2] https://review.gluster.org/#/c/16490/ , Adding missing FOPs

[3] https://review.gluster.org/#/c/16547/ , Adding the compound FOP to 
generator file.

[4] https://review.gluster.org/#/c/16586/ , Adding Code-gen framework to 
io-stats translator


Thank you all,

Menaka M.

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Outreachy] Results of IOZone and smallfile bench tests with some changes.

[Shyam]

On 10/15/2016 09:14 PM, Menaka Mohan wrote:

Hello,


I have been able to successfully run the IOZone bench test on the
Gluster setup. Kindly find the results here [1].


Neat, thanks.




But for the smallfile distributed I/O benchmark, the test fails. Since
we are taking 5 samples in each operation, the " --operation create "
fails reporting that the file already exits from the second
sample [1462.758118, None, None, None, None] . Necessary changes are to
be made such that the --operation create happens only once and others (
--operation read and --operation ls-l ) run as usual. Kindly correct me,
If I am wrong.


I am wondering why [3] is not getting triggered and cleaning up the 
files in your test run. It seems to be doing the job in my setup. 
Possibly something to check/debug when you get a chance.





Assuming that as the reason, I have made the necessary changes to the
code [sample _size for create operation is 1] and ran the bench test.
Kindly find the results here [2].


This should be fine for the exercise. The interest in running a test 
more than once is to ensure we do not have any noise from a single run 
that gives us incorrect numbers as a result. So for now, running this 
once is fine, unless you get some time to debug as above.





I will further work on it as a single script and post the output as
mentioned in the GitHub page. Since this is a very time consuming
process, I will simultaneously work on the next task which is to
investigate the tools to use and analyze the system resource usage
across the setup.


I would encourage looking at the next part (i.e tools to analyze the 
system) of this task, as that can provide some more meaningful data for 
the project.





[1]   https://paste.fedoraproject.org/451045/55349814/
[2]   https://paste.fedoraproject.org/451953/14765795/


[3] GlusterBench line that should cleanup small files: 
https://github.com/gluster/gbench/blob/master/bench-tests/bt--0001/GlusterBench.py#L257

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] [Outreachy] Results of IOZone and smallfile bench tests with some changes.

[Menaka Mohan]

Hello,


I have been able to successfully run the IOZone bench test on the Gluster 
setup. Kindly find the results here [1].


But for the smallfile distributed I/O benchmark, the test fails. Since we are 
taking 5 samples in each operation, the " --operation create " fails reporting 
that the file already exits from the second sample [1462.758118, None, None, 
None, None] . Necessary changes are to be made such that the --operation create 
happens only once and others ( --operation read and --operation ls-l ) run as 
usual. Kindly correct me, If I am wrong.


Assuming that as the reason, I have made the necessary changes to the code 
[sample _size for create operation is 1] and ran the bench test. Kindly find 
the results here [2].


I will further work on it as a single script and post the output as mentioned 
in the GitHub page. Since this is a very time consuming process, I will 
simultaneously work on the next task which is to investigate the tools to use 
and analyze the system resource usage across the setup.


[1]   https://paste.fedoraproject.org/451045/55349814/
[2]   https://paste.fedoraproject.org/451953/14765795/


Regards,

Menaka M
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Outreachy] Need help with running bench test on Gluster cluster.

[Shyam]

On 10/13/2016 01:54 AM, Menaka Mohan wrote:

Hello Shyam,



Thanks a lot for such a detailed reply. I am new to this and was trying
hard to learn and got your help in the right time. After getting your
reply, I have experimented more and following are the findings.



I am clear with all the pre-requisites except only one. I searched for
any -easy to understand- examples online, but couldn’t find any. My
clients.ioz file has the “hostname  directory iozone_pathname”. But
where to specify the number of threads per client ? In the recent
commit, Number of threads are set to 4 in the script. But how to do that
while running our own iozone command ?


So, the number of lines in the clients.ioz should equal to,  * 
. So say 'CLIENTS=hosta\ hostb', and threads are 4 (as in the 
script), then clients.ioz would have,


hosta  
hosta  
hosta  
hosta  
hostb  
hostb  
hostb  
hostb  

This should get things in order for you I believe.





So far, I am unable to find the cause of the following errors.



1)  extract_iozone_resuslt function returning None

2)  Line 13 in sync-drop-caches.sh “ source=${BASH_SOURCE[0]}”
returns Bad substitution.



But I am able to run the iozone commands in the script on my own and get
the results.

1)  # IOZone seq writes
[https://paste.fedoraproject.org/449217/14763288/]

2)  # IOZone seq reads
[https://paste.fedoraproject.org/449218/14763289/]

3)  # IOZone rand read / write
[http://paste.fedoraproject.org/449220/76329399/]


These look fine, except you are running with '-t 1', so if you correct 
the clients.ioz, then you should be able to run these with '-t 4', which 
would be the first thing to test.






Since we are getting the expected results, I assume covering all the
pre-requisites rightly except the above mentioned one.



While I run the actual gbench script, I get the following log. [
https://paste.fedoraproject.org/449225/63310741/ ]


The above, if you notice, is not getting any result from iozone, which 
is why the parser is failing to find 'Children see throughput' in the 
output, and you end up with the default result of "None".


I think if the clients.ioz is corrected, this should not occur. As 
stated earlier, let's get that fixed and see what results.


FWIW, the clients.ioz requirement to have as many lines per client is 
mentioned here [1], but possibly not very clearly (maybe another patch 
to the documentation :) ).


I also ran with iozone version 429 and a single line in my clients.ioz, 
the bench script failed, although the iozone output was a little more 
than what you see in your run. Let's hope that is some artifact of the 
setup and see if current suggestions work.




As I have checked, I also have bash shell as the default shell. [
https://paste.fedoraproject.org/449224/30577147/ ]



I am also trying to find the cause of the errors to my knowledge and
will report any progress soon. I have made few syntactic corrections in
the code and also updated the readme as you said in the previous email
and generated a pull request. Kindly review it.


Done, comments posted in github pull request.





Now, Please let me know what all you expect in the application other
than my understanding and the timeline and few initial contributions.


I would expect some results (assuming we are able to get this running 
and in order) and some analysis of any system resource across the setup 
(say memory or CPU or disk IO etc.), where you would need to investigate 
tool(s) to use and provide the results.






Note: I looked into few articles to reply properly into a thread in a
mailing list. I changed the subject as Re: Re: Sub. Still, If I am
unable to place it rightly in the thread, I am sorry. I will look into it.



Regards,

Menaka M


[1] clients.ioz configuration, see iozone section in this page, 
https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Performance%20Testing/


Or, search for "ioz.cfg contains these records" in the page above.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Outreachy] Need help with running bench test on Gluster cluster.

[Menaka Mohan]

Hello Shyam,

Thanks a lot for such a detailed reply. I am new to this and was trying hard to 
learn and got your help in the right time. After getting your reply, I have 
experimented more and following are the findings.

I am clear with all the pre-requisites except only one. I searched for any 
-easy to understand- examples online, but couldn't find any. My clients.ioz 
file has the "hostname  directory iozone_pathname". But where to specify the 
number of threads per client ? In the recent commit, Number of threads are set 
to 4 in the script. But how to do that while running our own iozone command ?

So far, I am unable to find the cause of the following errors.


1)  extract_iozone_resuslt function returning None

2)  Line 13 in sync-drop-caches.sh " source=${BASH_SOURCE[0]}" returns Bad 
substitution.

But I am able to run the iozone commands in the script on my own and get the 
results.

1)  # IOZone seq writes [https://paste.fedoraproject.org/449217/14763288/]

2)  # IOZone seq reads [https://paste.fedoraproject.org/449218/14763289/]

3)  # IOZone rand read / write 
[http://paste.fedoraproject.org/449220/76329399/]

Since we are getting the expected results, I assume covering all the 
pre-requisites rightly except the above mentioned one.

While I run the actual gbench script, I get the following log. [ 
https://paste.fedoraproject.org/449225/63310741/ ]
As I have checked, I also have bash shell as the default shell. [ 
https://paste.fedoraproject.org/449224/30577147/ ]

I am also trying to find the cause of the errors to my knowledge and will 
report any progress soon. I have made few syntactic corrections in the code and 
also updated the readme as you said in the previous email and generated a pull 
request. Kindly review it.

Now, Please let me know what all you expect in the application other than my 
understanding and the timeline and few initial contributions.

Note: I looked into few articles to reply properly into a thread in a mailing 
list. I changed the subject as Re: Re: Sub. Still, If I am unable to place it 
rightly in the thread, I am sorry. I will look into it.

Regards,
Menaka M


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Outreachy] Need help with running bench test on Gluster cluster.

[Shyam]

Took the liberty of adding [Outreachy] to the subject, makes it easier 
for others to search as well (probably). If people have objections, let 
me know and I will post a single response here removing the same for the 
future.




On 10/11/2016 03:53 PM, Menaka Mohan wrote:

Hi,


I am Menaka M. I am interested in participating in this round of
Outreachy under Gluster. I am new to this open source world. Kindly help
me with the following query.


Welcome!




I have set up the Gluster development environment with two servers and
one client. I am trying to run the basic bench test on the Gluster
cluster from this GitHub repo
. I
also have IOZone installed. While trying to run the provided script, i
get the following error. I was trying to identify the cause of the
error. Kindly help me with that.


It looks like you have made progress (from your IRC ping on clients.ioz 
and its contents). I guess you got the CLIENTS, SERVERS and some other 
prerequisites right. It also looks like you either figured out how to 
setup rsh for iozone, or exported RSH env-var as 'ssh'.  So some good 
progress there.


It would be nice if you could take some time out to add to the README 
these additional prerequisites. (when you get the time) (you would be 
looking at github pull requests [1] for the same)




So, I learned more about IOZone and also the performance testing section
in the Gluster docs. With that knowledge and to learn more, I have gone
through the code and running iozone commands mentioned in the
GlusterBench.py script individually.

If I am asking a very basic thing, apologies. I will quickly learn things.


Nope! not necessarily basic, as I got some failures too when using 
latest iozone binaries/sources. One such failure was in [2] where the 
latest iozone results are changed to report "kB/...", rather than 
"KB/..." and so the parsing failed. Anyway, I made a local fix to my 
python script for the same (will push the change out soonish).


So coming to your test failure:

- Could you post your full log using something like fpaste, and share 
the link (helps not making any assumptions on my part) [3]


- I hit a similar failure in my tests, see [4]. This run seems to have 
hit some ssh connection issue, because of which a sample was left as 
"None" (which is the default)


- In your case *all* samples are left as "None". So I suspect a more 
generic parsing failure of the results as obtained from *every* iozone run


- To detect what failed and why, it would be better to take a look at 
the entire log


- Additionally, you could also add a few prints within the 
extract_iozone_result function to debug the root cause of failure




Regards,
Menaka M

--

python GlusterBench.py -s 64 -f 1 -n 5 -v

Number threads = 4
Client list = HadoopSlave4

Running IOZone with 64KB record size and 4 threads,  Creating an 8 GB
file with every thread.
Running smallfile with 64KB files, creating 1 files.
Running squential IOZone tests, starting with sequential writes.

About to gather sample --> 0

Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
 Al Slater, Scott Rhine, Mike Wisner, Ken Goss
 Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
 Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
 Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,


Were there any additional output lines here? I do understand that you 
have kept the output terse later, but to understand parsing errors, the 
output from iozone would be helpful.




Adding the current sample to the list: None


The above line is where the parsing has failed, it should retrieve a 
number and not a default string "None" from the routine.



Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution


I think the default shell in your environment is not bash, and that 
leads to the above "Bad substitution". On a hunch I am providing a stack 
overflow link for the same [5]. This should not have a repercussion on 
the test result, but I could be mistaken.



Cleaning up files.

About to gather sample --> 1

/* I removed the Iozone and Contributors header */

Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
Cleaning up files.

About to gather sample --> 2

Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
Cleaning up files.

About to gather sample --> 3

Adding the current sample to the list: None
Dropping cache