[Gluster-users] Simple recipes for JBOD and AutoScaling.

more concrete examples. Thanks ! -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] forge wiki : Upgrades ?

Hi folks. Any chance of upgrading any features on the forge wiki over time ? i.e. adding images or trac style plugins etc etc ? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http

[Gluster-users] New upstream glusterfs-hadoop release 2.1.10

so that they test 2.0 specs, avoiding conflicts where 1.0 and 2.0 FS semantics differ (testListStatus). Happy hadooping ! The plugin jars can be found here : http://rhbd.s3.amazonaws.com/maven/indexV2.html -- Jay Vyas http://jayunit100.blogspot.com

[Gluster-users] quick hadoop on gluster update

you need it in your DFS. -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] BigPetStore on GlusterFS

finally cobbled together a somewhat polished demo of BigPetStore hadoop app on glusterfs. Anyone interest in running hadoop applications on top of gluster should definetly check it out! https://www.youtube.com/watch?v=OVB3nEKN94k -- Jay Vyas http://jayunit100.blogspot.com

Re: [Gluster-users] linux flash filesystems and GlusterFS

Are there any other requirements other than xattr support? Would be cool to find ever kernel fs impl and test them automatically on a server (maybe our rack space nodes) somewhere for a series of known gluster requirements. On Apr 2, 2014, at 6:49 AM, Apostolos Manolitzas

Re: [Gluster-users] PLEASE READ ! We need your opinion. GSOC-2014 and the Gluster community

I vote for 3,2,1 : Best: the accelerator looks cool and would be awesome for supporting hi throughput workloads w/ different consistency gaurantees. In hadoop this could be valuable I think when we want fast ETL but can wait a few seconds for files to b available in global namespace 2nd

[Gluster-users] Has anyone dockerized gluster yet?

for gluster on fedora/centos/... Has anyone set up a docker container that installs and mounts a couple of gluster peers as linux containers yet? It love to see how that works, and then maybe layer in hadoop on top of it. Jay Vyas http://jayunit100.blogspot.com

Re: [Gluster-users] gluster on VM and RDBMS

://supercolony.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com

[Gluster-users] getfattr in complex deployments

, regradless of the replication domain? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Atomic file updates

For vanilla apps that are doing stuff in gluster, you normally do it through a fuse mount. mount -t glusterfs localhost:HadoopVol /mnt/glusterfs But in your case, you might want to do some strict consistency settings to make it atomic: mount -t glusterfs localhost:HadoopVol -o

Re: [Gluster-users] Atomic file updates

Glass t...@tmgcon.com wrote: On 02/13/2014 10:24 AM, Jay Vyas wrote: For vanilla apps that are doing stuff in gluster, you normally do it through a fuse mount. mount -t glusterfs localhost:HadoopVol /mnt/glusterfs But in your case, you might want to do some strict consistency settings

[Gluster-users] libgfapi consistency model

Hi gluster ! How does libgfapi enforce FileSystem consistency? Is it better than doing this than exsiting FUSE mounts which require the *timeout parameters to be set to 0? Thanks! This is important to us in hadoopland. -- Jay Vyas http://jayunit100.blogspot.com

Re: [Gluster-users] libgfapi consistency model

mode. This approach is very similar to the retry based optimistic path resolution in the more recent linux kernel vfs. HTH Avati On Wed, Feb 5, 2014 at 8:31 AM, Jay Vyas jayunit...@gmail.com wrote: Hi gluster ! How does libgfapi enforce FileSystem consistency? Is it better than doing

Re: [Gluster-users] gluster CPU cycles at 100%

okay... :) .. but whats a gluster statedump? On Tue, Jan 28, 2014 at 2:08 PM, Harshavardhana har...@harshavardhana.netwrote: You should take a 'gluster' statedump of the process - and open a bugzilla for analysis with logs. On Tue, Jan 28, 2014 at 10:30 AM, Jay Vyas jayunit

Re: [Gluster-users] gluster CPU cycles at 100%

bump ^^ whats a gluster statedump and how do I create one? this 100+% CPU issue is very intriguing, and id like to create a bug for it (if it indeed is a bug) On Tue, Jan 28, 2014 at 3:07 PM, Jay Vyas jayunit...@gmail.com wrote: okay... :) .. but whats a gluster

Re: [Gluster-users] gluster CPU cycles at 100%

ah nevermind, googled it: shoulda done this earlier. On Tue, Jan 28, 2014 at 4:58 PM, Jay Vyas jayunit...@gmail.com wrote: bump ^^ whats a gluster statedump and how do I create one? this 100+% CPU issue is very intriguing, and id like to create a bug for it (if it indeed is a bug

Re: [Gluster-users] gluster CPU cycles at 100%

at 10:30 AM, Jay Vyas jayunit...@gmail.com wrote: Hi folks : Im running mahout on top of gluster using the GlusterFileSystem hadoop plugin. It works well, but Im noticing that glusterfsd is at 100% CPU, and a copy action seems to be stalling after a long workload. Any ways to rescue

Re: [Gluster-users] Automatically Deploying GlusterFS with Puppet-Gluster+Vagrant!

http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Forge wiki pages are to narrow

hi johnmark: I think the glusterforge wiki pages are WAY to narrow for sophisticated docs Can you fix them so that wiki pages can be shorter and easier to read without as much scrolling? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster

Re: [Gluster-users] puppet-gluster from zero: hangout?

...@gmail.com wrote: On Sun, 2013-12-15 at 13:32 -0500, John Mark Walker wrote: Let's send a reminder to jmwbot so It's on my calendar :) Ahh! Super glad you saw it ;) -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users

Re: [Gluster-users] puppet-gluster from zero: hangout?

Google hangout. I don't have google hangout software installed on my computer. -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] puppet-gluster from zero: hangout?

me if im wrong, but if so ~ Let me propose a date: Tuesday at 12 EST (thats 5 PM in london - which i think should work for james as well as us out here on the other side of the pond). -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing

[Gluster-users] Are directory moves cheap?

Hi gluster! Are move/rename ops over directories in fuse mounted gluster generally fast when there are TBs of data in the underlying dirs? ...Context follows... One of the common operations in hive data manipulation is load data, where you move a set of files from one place to another when

Re: [Gluster-users] Gluster Community Weekly Meeting

list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster

Re: [Gluster-users] A question about comparing gluster to other storage

-- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] in honor of fredrick sanger

of genomic or bioinformatics data. http://www.genengnews.com/gen-news-highlights/frederick-sanger-father-of-genomic-era-dies-at-95/81249136/ http://www.eaglegenomics.com/2013/03/glusterfs-vs-a-future-distributed-bioinformatics-file-system/ anyways, just a thought -- Jay Vyas http://jayunit100

Re: [Gluster-users] Gluster KVM scripts

incantations you've been working on ! On Sun, Nov 17, 2013 at 6:28 PM, James purplei...@gmail.com wrote: On Sun, 2013-11-17 at 17:29 -0500, Jay Vyas wrote: hi james, moving this to public. Moved to gluster-users then. The subject was how to start using jame's puppet modules

[Gluster-users] Large files.. What's the latest news?

Hi folks. The precious question regarding xfs got me thinking.. Any news on these two questions or existing modifications to gluster underway to deal with larger-than-brick files? 1) Are bricks aware if the total underlying file system size? 2) And when we try to write a file that is hashed

Re: [Gluster-users] Gluster Cheat Sheet

+1 for cheat sheets. i always like when these things are embedded in the actual README, or else, somewhere else in the code, because they stay up to date better that way. ___ Gluster-users mailing list Gluster-users@gluster.org

Re: [Gluster-users] Cassandra on Gluster

to the Viddler blog http://blog.viddler.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com

[Gluster-users] metadata for stat : Should it be identical?

Hi folks. Im wondering wether i should expect gluster to always return identical stat info for a file on the system. seems like, by definition of a distributed file system the answer would be yes. But Im not seeing that. Rather, I'm seeing different metadata from the stat command, in

Re: [Gluster-users] metadata for stat : Should it be identical?

http://www.pool.ntp.org/en/**use.htmlhttp://www.pool.ntp.org/en/use.html On 10/24/2013 12:14 PM, Jay Vyas wrote: FYI, these two machines are not clock synchronized, but -- should that be an issue? -- Alex Chekholko ch...@stanford.edu 347-401-4860

Re: [Gluster-users] New to GlusterFS

So Im sure it would work for you the same as any distributed fs would , and you could always optimize things later if it wasn't fast enough. Gluster is very tune able. I'm curious -- do you know what your access patterns are going to be? Is this for testing or for a real production system?

Re: [Gluster-users] A Wizard for Initial Gluster Configuration

-users mailing list Gluster-users@gluster.org http://supercolony.gluster.**org/mailman/listinfo/gluster-**usershttp://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list

Re: [Gluster-users] (no subject)

, 2013, at 8:40 PM, James purplei...@gmail.com wrote: On Thu, 2013-10-03 at 14:14 -0400, Jay Vyas wrote: FYI If youre interested in Trying to play with a gluster distribtued set up on VMs, you can try to spin up and have vagrant installed , Checkout this post : http://www.gluster.org/2013/10

Re: [Gluster-users] (no subject)

thanks james :) how about a INSTALL_FROM_ZERO instructions in the readme? :) On Fri, Oct 4, 2013 at 10:31 AM, James purplei...@gmail.com wrote: On Fri, Oct 4, 2013 at 9:36 AM, Jay Vyas jayunit...@gmail.com wrote: Ah yes thanks James..! Forgot about this - haven't adopted it yet because I

[Gluster-users] peer probe stochastic behaviour in shell

returns NOTHING. the third call returns SUCCESS. Any thoughts on this? FYI afterwards, I can do peer probe / peer detach in tandem 30 times with no errors, so im maybe there can be a cold start issue with probing? Also FYI , these are in F19 VMs which have run iptables -F -- Jay Vyas http

Re: [Gluster-users] (no subject)

-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] set up a Gluster 2 node cluster in two lines of code :)

would like if we could all share and centralizes out virtualization utilities somewhere, hope this provides us with a start! On Sep 30, 2013, at 11:38 AM, Justin Clift jcl...@redhat.com wrote: On Sun, 29 Sep 2013 11:18:37 -0400 Jay Vyas jayunit...@gmail.com wrote: snip Just finished

[Gluster-users] set up a Gluster 2 node cluster in two lines of code :)

. - bug reporting - testing functionality of different tuning/config options - testing hadoop interop .. Will blog post about it soon , but till then just ping me if you want to get involved etc.. -- Jay Vyas http://jayunit100.blogspot.com

Re: [Gluster-users] Enabling Apache Hadoop on GlusterFS: glusterfs-hadoop 2.1 released

Thanks for the announcement steve ! FYI ~ for those interested, there are alot of different areas of expertise which can be useful in this project. I've added a list here: https://forge.gluster.org/hadoop/pages/HackIdeas . In my view, people from the Gluster, DevOps, Linux , and MapReduce

Re: [Gluster-users] Gluster to support scalable web site

on the topic you might like to share. ** ** Thanks in advance for any help you might be able to provide Michael ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay

[Gluster-users] higher op version

source on all servers. Example.. [root@hbase-regionserver3 ~]# gluster peer probe hbase-head peer probe: failed: Peer hbase-head is already at a higher op-version [root@hbase-regionserver3 ~]# exit [root@hbase-master ~]# gluster peer probe hbase-regionserver3 peer probe: success -- Jay Vyas http

Re: [Gluster-users] Gluster version....

Ah okay. May I suggest bundling the commit id or some other unambiguous information into the source build ? On Jul 18, 2013, at 7:42 AM, Kaleb KEITHLEY kkeit...@redhat.com wrote: On 07/17/2013 09:56 PM, Jay Vyas wrote: hi gluster : Okay so ive been playing around in my vm, and wanted

[Gluster-users] Gluster version....

it simply doesnt tag the version when you build from source? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] One Volume Per User - Possible with Gluster?

Hmmm... but given that glusters fuse client is posix compliant, can't you just create a single volume and use a customized umask setup on user-named subdirectories in that volume to mimic this behaviour? On Jul 2, 2013, at 7:25 PM, Joshua Hawn j...@picloud.com wrote: I've been looking into

Re: [Gluster-users] Gluster KVM scripts

Hi james: I didnt know were behind this :) I saw it the other day .. I guess i better play some with https://forge.gluster.org/puppet-gluster to see whats available and maybe ill post directly here or leave feedback on glusterforge ___ Gluster-users

[Gluster-users] local caching of file across global cluster

heuristics for us... and its just a matter of time for FUSE/Underlying filesystem/Gluster mount to figure out that a file is important before it starts caching it in some magical sort of way. -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users

[Gluster-users] --fopen-keep-cache glusterfs

Thanks brian for the response regarding caching. What exactly does the --fopen-keep-cache glusterfs option to and are there any docs on it? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http

Re: [Gluster-users] Glusterfs-Hadoop

Hi and thanks for playing with the GlusterFS hadoop plugin. It is under active development, And We have a more up to date plugin with several fixes, you can grab it our staging glusterfs-hadoop release server (ill forward the link to you shortly..) Or else, easily build from source using the

Re: [Gluster-users] Glusterfs-Hadoop

! ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http

[Gluster-users] peer probe fails (107)

with the net.ipv4.ip_nonlocal_bind parameter, and also a bug https://bugzilla.redhat.com/show_bug.cgi?id=890587 ... But I'm still not sure what the nature of this error is - any thoughts? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing

Re: [Gluster-users] peer probe fails (107)

On Mon, May 20, 2013 at 3:45 PM, Jay Vyas jayunit...@gmail.com wrote: Hi gluster: Im getting the cryptic 107 error, (I guess this means gluster can't see a peer)... gluster peer probe vm-2 peer probe: failed: Probe returned with unknown errno 107 When I can effectively ssh and ping

[Gluster-users] NoSQL tools that run on Gluster

thought experiment? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] NoSQL tools that run on Gluster

...@redhat.com wrote: On 05/01/2013 01:50 PM, Jay Vyas wrote: There has been chatter about X on gluster, where x=mongo, riak,... Im wondering - is there a most popular or most well tested transactional datastore that runs/leverages gluster ? Or is the idea of running a transactional

Re: [Gluster-users] NoSQL tools that run on Gluster

or special translators can add even more features and performance. Thanks, Greg - Original Message - From: Jeff Darcy jda...@redhat.com To: Jay Vyas jayunit...@gmail.com Cc: Gluster-users@gluster.org gluster-users@gluster.org Sent: Wednesday, May 1, 2013 11:26:47 AM

Re: [Gluster-users] Gluster and Cloudera's Hadoop

https://bugzilla.redhat.com/show_bug.cgi?id=951305 Hope this helps. On Mon, Apr 8, 2013 at 6:07 PM, Jay Vyas jayunit...@gmail.com wrote: Hi james: Looks like standard Hadoop seems to want to keep the files as permission 700, just like you mention in your email: https://svn.apache.org

Re: [Gluster-users] How to properly shutdown Gluster (glusterd *and* related processes)

___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users

[Gluster-users] Gluster and Cloudera's Hadoop

github repository: https://github.com/gluster/hadoop-glusterfs, where these fixes have been merged into head. Also we can get you this jar prebuilt if you want, just let me know... Thanks for trying out the GlusterFileSystem and keep the feedback coming ...! -- Jay Vyas http://jayunit100

Re: [Gluster-users] Gluster and Cloudera's Hadoop

if you try submitting the job from the same machine that is running your jobtracker? I've seen this error before when submitting jobs from random places. Again, the above is more of a guess than anything else, until we look further into it. On Mon, Apr 8, 2013 at 4:01 PM, Jay Vyas jv

Re: [Gluster-users] A simple way to federate gluster namespace ?

unionfs sounds like it may work. Not sure what you mean by tree? On Fri, Apr 5, 2013 at 6:56 PM, Robert Hajime Lanning lann...@lanning.ccwrote: On 04/05/13 15:33, Jay Vyas wrote: Hi guys: BTW thanks for the insights regarding locality . Now I have a new stupid question for you

[Gluster-users] A simple way to federate gluster namespace ?

/subA /submount/a mount -o /tmp/supermount/subB /submount/b Or maybe you could do the equivalent with symlinks? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman

[Gluster-users] GlusterFS and optimization of locality.

access file2. Server 3 will access all ten files (the whole data set). Is there a way to get gluster to rebalance bricks over time based on access patterns ... or otherwise .. what is the best way to increase the average locality of access to files in the cluster ? -- Jay Vyas http://jayunit100

Re: [Gluster-users] MapReduce on glusterfs in Hadoop

Hi nithin: The fuse mount is what allows the filesystem to access distributed files in gluster: that is, GlusterFS has its own fuse mount ... And GlusterFileSystem wraps that in hadoop FileSystem semantics. Meanwhile, The mapreduce jobs are invoked using on custom core-site and mapred-site

[Gluster-users] (no subject)

Hi guys: Oddly, when I try to start glusterd using sudo service glusterd start I get no logs to /var/log/gluster* Upon jeff's advice, I ran glusterd this way: glusterd --debug -f /etc/glusterfs/... And below is the output. It appears to be related to ports but im not sure ... any idea whats

[Gluster-users] libgfapi docs

Hi guys: Whats the status for documentation on the libgfapi. -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] compile error (glusterfs)

-recursive] Error 1 -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Failed to perform brick order check,...

, and why does brick order matter? -- Jay Vyas http://jayunit100.blogspot.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] mount glusterfs: ports

+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down [2012-12-30 20:44:17.536160] I [fuse-bridge.c:4643:fini] 0-fuse: Unmounting '/mnt/petshop'. Jay Vyas http://jayunit100.blogspot.com -- Jay Vyas http://jayunit100.blogspot.com