Big Top and Ecosystem coverage goals...

2013-06-10 Thread Jay Vyas
there a single web page that lists all of the HCatalog ecosystem test coverage? If not maybe I can dig through the source and compile one for the site. -- Jay Vyas http://jayunit100.blogspot.com

The combinatorial problem of testing hadoop ecosystem tools

2013-06-10 Thread Jay Vyas
fact that ecosystem tools are constantly being upgraded at different rates. -- Jay Vyas http://jayunit100.blogspot.com

BigTop : Build Targets

2013-06-10 Thread Jay Vyas
Hi again Big Top ! What are the "goals" as far as software artifacts coming out of bigtop ? I see that in the jenkins server there are all sorts of things, everything from - specific versions of tools which are considered part of the ecosystem (i.e. the flume rpm) http://bigtop01.cloudera.org:8

Re: BigTop : Build Targets

2013-06-10 Thread Jay Vyas
thanks sean: To augment, I found this presentation http://www.docstoc.com/docs/155631546/1615-Bigtop-presentation, where the goals of bigtop are also stated: - Generates packages of ecosystem tools - Generates "convenience artifacts" for certain tools - Runs Integration tests of tools So I guess

Re: BigTop : Build Targets

2013-06-10 Thread Jay Vyas
Thanks roman :) yes we once lived and died by the SolrOutputFormat for some time. It is a very nice extension to hadoop's reduce outputs - but What i mean is that SOLR is not part of the hadoop ecosystem, in the sense that it doesnt natively depend on HDFS . Rather it uses standard file system a

Re: BigTop : Build Targets

2013-06-10 Thread Jay Vyas
-deploy/puppet/ should be https://github.com/apurtell/bigtop/tree/master/bigtop-deploy -- On Mon, Jun 10, 2013 at 6:23 PM, Jay Vyas wrote: > Thanks roman :) yes we once lived and died by the SolrOutputFormat for > some time. It is a very nice extension to hadoop's reduce outputs - b

Re: BigTop : Build Targets

2013-06-10 Thread Jay Vyas
venture to say that by this definition, SOLR is not an ecosystem tool. On Mon, Jun 10, 2013 at 7:41 PM, Roman Shaposhnik wrote: > On Mon, Jun 10, 2013 at 3:23 PM, Jay Vyas wrote: > > Thanks roman :) yes we once lived and died by the SolrOutputFormat for > some > > time.

Re: BigTop : Build Targets

2013-06-10 Thread Jay Vyas
ll be some work integrating it into our build system and trunk." On Mon, Jun 10, 2013 at 8:09 PM, Jay Vyas wrote: > @Roman: This is now a thread about SOLR more so than bigtop, so apologies > for being off topic, but I see no dependency in SOLR on hadoop: > > git clone git://github.com/ap

Is HBase embedded on BigTop VMs?

2013-06-11 Thread Jay Vyas
ResourceManager 1327 SecondaryNameNode 2786 JobHistoryServer 1213 NameNode I would assume that bigtop would deploy hbase in the VMs -- Jay Vyas http://jayunit100.blogspot.com

Re: Is HBase embedded on BigTop VMs?

2013-06-12 Thread Jay Vyas
> > On Tue, Jun 11, 2013 at 9:26 AM, Jay Vyas wrote: > > It appears that the VM here: > > > > > http://bigtop01.cloudera.org:8080/job/Bigtop-VM-matrix/BR=master,KIND=kvm,label=fedora16/lastSuccessfulBuild/artifact/bigtop-vm-kvm-master.tar.gz > > > > Does

Running tests without VM / install context

2013-06-17 Thread Jay Vyas
Hi : Is there a distilled example of how to run the bigtop smoke tests outside of the context of the bigtop framework? For example, on a live cluster - how can I run the hive smoke tests? -- Jay Vyas http://jayunit100.blogspot.com

Hive/HCatalog/Pig/etc.. smoke tests vs full integration tests

2013-06-18 Thread Jay Vyas
el of integration tests for individual ecosystem tools in BigTop? i.e. is there a full set of integration tests that confirm, for example, that hive is correctly working on my cluster? Or is the hive smoke test sufficient? -- Jay Vyas http://jayunit100.blogspot.com

VM with smoke tests that pass?

2013-06-29 Thread Jay Vyas
nd all the ecosystem is installed.) This is pretty important for development i think. -- Jay Vyas http://jayunit100.blogspot.com

Re: VM with smoke tests that pass?

2013-06-30 Thread Jay Vyas
; > That should be quite easy to follow. > Cos > > On Sat, Jun 29, 2013 at 09:51PM, Jay Vyas wrote: >> Hi big top : >> >> I've been working on a recent big top VM and am finding that many required >> components for smoke tests (i.e. mysql, snappy, ...)

SQOOP_URL

2013-07-01 Thread Jay Vyas
n the code anywhere. -- Jay Vyas http://jayunit100.blogspot.com

Re: SQOOP_URL

2013-07-01 Thread Jay Vyas
Jul 1, 2013 at 4:00 AM, Jay Vyas wrote: > >> Hi : Im finding that the pom.xml for the sqoop smoke tests requires a > >> "SQOOP_URL": > >> > >> > >> org.apache.maven.plugins > >> enforce-property > &

Why are environmental variables necessary for building jars?

2013-07-03 Thread Jay Vyas
Hi bigtop! The top level pom from the smoke tests --> ../smokes/../common/pom.xml Requires the HADOOP_HOME and HADOOP_CONF_DIR But, it seems as though we should be able to build the smoke jars with or without these variables set, correct? -- Jay Vyas http://jayunit100.blogspot.com

permanant builds on jenkins

2013-07-06 Thread Jay Vyas
n the DataNode path, and also, my namenode appears to really like being in safe mode. these could just be due to VM setup though, as im changing some things like adding static IPs and data node write paths... so nothing to be alarmed about.) -- Jay Vyas http://jayunit100.blogspot.com

Re: permanant builds on jenkins

2013-07-07 Thread Jay Vyas
t least, are working. If we could get them working with static IPs then we could even clone down the bigtop source code and run bigtop hadoop tests on the VMs after construction as a validation step. On Sat, Jul 6, 2013 at 8:58 PM, Bruno Mahé wrote: > On 07/06/2013 07:33 AM, Jay Vyas wrote: &g

Building and deploying a custom bigtop iTest suite?

2013-07-10 Thread Jay Vyas
build :( I guess I might be interpretting the README wrong... but how do I install my local bigtop source code into my local maven repo, so that I can run customized bigtop groovy (iTest) tests? Probably this is more of a pom.xml question than a bigtop one. Thanks! -- Jay Vyas http

Re: Building and deploying a custom bigtop iTest suite?

2013-07-11 Thread Jay Vyas
ings.xml or your custom > pom.xml file (something along the line of not using correct artifact > versions, > etc.) than a bigtop issue. The scenario you're describing is working > everytime I've tried. > > Cos > > On Wed, Jul 10, 2013 at 07:10PM, Jay Vyas wrote: >

Blog post on customizing bigtop

2013-07-16 Thread Jay Vyas
Hi bigtop ! Hope this is useful. I wrote up how Im going about customizing our bigtop tests. http://jayunit100.blogspot.com/2013/07/customizing-apache-big-top-smoke-tests.html -- Jay Vyas http://jayunit100.blogspot.com

Re: Blog post on customizing bigtop

2013-07-16 Thread Jay Vyas
ng it later on... thanks for posting. > Best > Lewis > > > On Tue, Jul 16, 2013 at 1:25 PM, Jay Vyas wrote: >> Hi bigtop ! >> >> Hope this is useful. I wrote up how Im going about customizing our bigtop >> tests. >> >> http://jayunit100

Re: old information in README file

2013-07-19 Thread Jay Vyas
correct instructions on the wiki, so maybe it may be > merged or > > referenced instead: > > > > > https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests > > > > Thank you > > > > Martin Bukatovic > > > > -- > *Lewis* > > -- Jay Vyas http://jayunit100.blogspot.com

Updated README

2013-07-19 Thread Jay Vyas
Hi bigtop! I submitted an updated README patch with simpler command invocations and corrected paths. Hope this helps ! https://issues.apache.org/jira/secure/attachment/12593243/README.patch -- Jay Vyas http://jayunit100.blogspot.com

Re: old information in README file

2013-07-19 Thread Jay Vyas
your patch to the issue you created in Jira. Folks can then apply > your > > patch and run with it. > > > > I would like for some Bigtop PMC's to jump in here as maybe a feature > branch > > for every patch is a bit of an overkill. > > > > hth >

Customizing pom behaviour across different modules

2013-08-23 Thread Jay Vyas
Hi bigtop: I have hacked a bigtop test cycle together which builds and tests from my source. It is a bit of a hack because: 1) It consists of 2 build phases first is online, to pull deps and install snapshots and second one offline, to overwrite snapshots with local source. 2) it modifies t

Terasort/Teragen in smokes

2013-08-27 Thread Jay Vyas
Hi guys: I run TeraSort/TeraGen as additions to bigtop in some shell scripts. Any interest in these as an update to TestHadoopExamples in the MapReduce smokes? If so I could patch them in :) let me know and I'll put in a JIRA ! -- Jay Vyas http://jayunit100.blogspot.com

Re: Terasort/Teragen in smokes

2013-08-27 Thread Jay Vyas
benchmarking is in the cards for bigtop? On Aug 27, 2013, at 7:39 PM, Roman Shaposhnik wrote: > On Tue, Aug 27, 2013 at 4:26 PM, Jay Vyas wrote: >> Hi guys: >> >> I run TeraSort/TeraGen as additions to bigtop in some shell scripts. >> >> Any interest in these as a

BinSedesTuple: Pig Smokes

2013-08-28 Thread Jay Vyas
not curate its own pigsmokes, but use the pig/ projects smoke tests forthe forseeable future? -- Jay Vyas http://jayunit100.blogspot.com

Translate this to english :)

2013-08-28 Thread Jay Vyas
In particular... 1) What is the difference between flume and flume ng - and does "flume_ng" support these options (i.e. node_nowatch) ? 2) What is the "node_config |" syntax accomplishing? The entire Flume bigtop smoke test can be found here: https://github.com/apache/bigtop/tree/master/bigtop-tests/test-artifacts/flume/src/main/groovy/org/apache/bigtop/itest/flumesmoke -- Jay Vyas http://jayunit100.blogspot.com

Re: BinSedesTuple: Pig Smokes

2013-08-28 Thread Jay Vyas
, 2013 at 5:53 PM, Mark Grover wrote: > +1 to what Roman said. > > And, Jay, could you please paste the error for our curiosity anyway? > > > > On Wed, Aug 28, 2013 at 2:49 PM, Roman Shaposhnik wrote: > >> On Wed, Aug 28, 2013 at 12:50 PM, Jay Vyas wrote: >

ITest Shell.groovy hangs

2013-08-28 Thread Jay Vyas
s a 1-element array on an // empty String if (baosErr.size() != 0) { err = baosErr.toString().split('\n'); } else { err = new ArrayList(); } println(key+" PROCESS::: wait for ") -- Jay Vyas http://jayunit100.blogspot.com

Re: ITest Shell.groovy hangs

2013-08-29 Thread Jay Vyas
Thanks roman. What is this patch doing ? how is it more defensive then the existing one? On Thu, Aug 29, 2013 at 1:07 PM, Roman Shaposhnik wrote: > On Wed, Aug 28, 2013 at 6:30 PM, Jay Vyas wrote: > > Hi bigtop ! Im finding that a complex shell script, run via Shell.groovy, > &g

Lightweight /miscellaneous user tests

2013-08-31 Thread Jay Vyas
one idea I had was to have a submodule for unrelated/miscellaneous "contrib" tests so that people could just add shell scripts into a resources directory and mvn verify would just run each as a test. The generic Test groovy file could just do an ls, and run each script and examine the exit co

A minor pig smokes patch

2013-09-03 Thread Jay Vyas
. Although I know bigtop is moving towards rpm-ish installation of stuff, the hardcoded dependency on /var/lib/ for pig's installation location imposes some unnecessary limitations on the way one can run the pig smokes . Anyways... happy hadooping . -- Jay Vyas http://jayunit100.blogspot.com

Whats a good venue to share bigtop examples ? blog? code? g+ hangout?

2013-09-04 Thread Jay Vyas
t in the bigtop community where we share the way we use bigtop to do different tasks and provide each other with feedback. -- Jay Vyas http://jayunit100.blogspot.co <http://jayunit100.blogspot.com>

Re: Terasort/Teragen in smokes

2013-09-05 Thread Jay Vyas
HBase. On Wed, Aug 28, 2013 at 12:02 AM, Konstantin Boudnik wrote: > On Tue, Aug 27, 2013 at 10:18PM, Jay Vyas wrote: > > hmm ok. now. Thinking about teragen makes me think of benchmarking.. > > > > In the longer term we could add benchmarking jobs to all the submodules >

Proposal: Stress tests submodule

2013-09-05 Thread Jay Vyas
Hi bigtop: Are there any filesystem stress tests? I.e -running 100 small mapreduce jobs -recursively deleting and recreating a directory multiple times from different clients, concurrently. -attempting to streaming io into the same file concurrently from different tasks I don't believe s

Re: Proposal: Stress tests submodule

2013-09-08 Thread Jay Vyas
Thanks roman ! https://issues.apache.org/jira/browse/BIGTOP-1065 <--- ive moved it to this jira thread. On Sat, Sep 7, 2013 at 10:15 PM, Roman Shaposhnik wrote: > On Thu, Sep 5, 2013 at 1:54 PM, Jay Vyas wrote: > > Hi bigtop: > > > > Are there any filesystem stress tes

Any thoughts on this new JIRA: re, mapred.* tests

2013-09-11 Thread Jay Vyas
possible, even if we know those tests are less than idea.? https://issues.apache.org/jira/browse/BIGTOP-1066 -- Jay Vyas http://jayunit100.blogspot.com

Ping regarding HCFS ticket

2013-09-16 Thread Jay Vyas
. -- Jay Vyas http://jayunit100.blogspot.com

Getting started with the puppet stuff

2013-09-19 Thread Jay Vyas
Does anyone have a recipe for packaging hadoop ecosystem components from the puppet modules already in bigtop? I assume it's more than just running "apply" and pasting them into the manifest, since ip addresses, turning services on/off, etc is involved for most of the ecosystem components..

Re: "bigpetstore" another idea that i forgot to mention which might fit into bigtop.

2013-09-19 Thread Jay Vyas
data somewhere on a public s3 instance , and have small input csv text files as a failsafe inside the repo so people can always run it from just the git repo alone. On Thu, Sep 19, 2013 at 5:28 PM, Roman Shaposhnik wrote: > On Thu, Sep 19, 2013 at 2:19 PM, Jay Vyas wrote: > > H

"bigpetstore" another idea that i forgot to mention which might fit into bigtop.

2013-09-19 Thread Jay Vyas
the hadoop ecosystem and useing bigtop as a venue to do so, an example app of this sort might be particularly useful. Apologies is this is off scope of bigtop but let me know! -- Jay Vyas http://jayunit100.blogspot.com

Re: "bigpetstore" another idea that i forgot to mention which might fit into bigtop.

2013-09-23 Thread Jay Vyas
bumtp^^ ... Any thoughts on where these blueprints should go and how to organize them? At that point ill roll it into a jira On Thu, Sep 19, 2013 at 5:41 PM, Jay Vyas wrote: > Okay that makes sense. Now time for my all-to-often asked bigtop > question: > > Where would thi

Request create new top level component: Blueprints

2013-09-24 Thread Jay Vyas
ot;blueprints". Seems like this is really a separate project in bigtop. -- Jay Vyas http://jayunit100.blogspot.com

New jiras (thanks) :)

2013-09-25 Thread Jay Vyas
Thanks for all these new jiras focused on end users roman / cons !!(documentation, simplifying hive, jenkins)... The hive one is sooo true: it's annoying that I need a hive server running to do smoke tests. Would like to just smoke test hive ETL without the extra stuff for simpler clusters. I

irc ?

2013-09-30 Thread Jay Vyas
Hi everyone: Is the community large enough for an IRC presence? I just checked out #bigtop on freenod but not much action there: "move along, nothing to see" :) ? -- Jay Vyas http://jayunit100.blogspot.com

Question on bigtop-deploy/vm/boxgrinder

2013-09-30 Thread Jay Vyas
I notice that bigtop-deploy/vm/boxgrinder has two files: bigtop_hadoop.appl bigtop-init-hdfs Why is it that the boxgrinder vm only starts up hdfs, why not all the mr2 services? -- Jay Vyas http://jayunit100.blogspot.com

Re: irc ?

2013-09-30 Thread Jay Vyas
Boudnik wrote: > Hi Jay. > > We are hanging there... once in a while I guess. I, personally, consider > IRC > or any other sort of indirect IMs, to be a huge hindrance on one's time. > But > that just me, I suppose. > > Cos > > On Mon, Sep 30, 2013 at 01:34PM

Re: Question on bigtop-deploy/vm/boxgrinder

2013-09-30 Thread Jay Vyas
> > On Mon, Sep 30, 2013 at 01:53PM, Jay Vyas wrote: > > I notice that bigtop-deploy/vm/boxgrinder has two files: > > > > bigtop_hadoop.appl > > bigtop-init-hdfs > > > > Why is it that the boxgrinder vm only starts up hdfs, why not all the mr2 > > se

Re: Question on bigtop-deploy/vm/boxgrinder

2013-09-30 Thread Jay Vyas
Okay so you are saying that the VM's generated from the appl description cannot be used unless HDFS is first running inside the vm, because they start YARN up, and YARN cannot start unless HDFS is running? If so, wouldnt if be more appropriate to have a startup script in the VMs which simple start

Re: irc ?

2013-10-03 Thread Jay Vyas
Speaking of: Any bigtop devs on the east coast to schedule a hackathon maybe in ny, ct, or ma? Or are you all out in California. > On Oct 2, 2013, at 7:23 PM, Roman Shaposhnik wrote: > >> On Mon, Sep 30, 2013 at 1:13 PM, Jay Vyas wrote: >> hi Cos: Its a good point, but i

How do the spec files get / reference source?

2013-10-05 Thread Jay Vyas
ce code is pulled in inside of these .spec files. This might also be a general .spec question, if so apologies in advance. -- Jay Vyas http://jayunit100.blogspot.com

Re: How do the spec files get / reference source?

2013-10-07 Thread Jay Vyas
Thanks roman ! Okay so, can you provide me with some guidance here: # Mahout MAHOUT_NAME=mahout MAHOUT_RELNOTES_NAME=Apache Mahout MAHOUT_PKG_NAME=mahout MAHOUT_BASE_VERSION=0.7 MAHOUT_PKG_VERSION=0.7 MAHOUT_RELEASE_VERSION=1 MAHOUT_TARBALL_DST=mahout-distribution-$(MAHOUT_BASE_VERSION)-src.tar.g

Setting up Dev environment for Test-Artifacts.

2013-10-08 Thread Jay Vyas
like this: Any way to add all the junit/and other maven dependencies in bigtop jars into the classpath at the eclipse level? Thanks! -- Jay

Why does test-artifacts have a "failsafe" integration test goal?

2013-10-08 Thread Jay Vyas
to build the jars for the smoke execution, not to run the integration tests ... ! Maybe im missing something :) if so let me know! Thanks ! -- Jay Vyas http://jayunit100.blogspot.com

A first glance/reminder/hack at the BigPetStore pipeline

2013-10-08 Thread Jay Vyas
alues. At that point the sample application will be ready and the first iteration of bigtop.blueprints will be ready to share. If Any initial thoughts or anyone else wants to jump in, let me know.? :) Jay Vyas http://jayunit100.blogspot.com

East coast bigtop hackday/microtalks; Any interest?

2013-11-12 Thread Jay Vyas
either location is valid for me... connecticut also would work (of course) as its central to both. :) -- Jay Vyas http://jayunit100.blogspot.com

Re: East coast bigtop hackday/microtalks; Any interest?

2013-11-12 Thread Jay Vyas
Artem Ervits > Data Analyst > New York Presbyterian Hospital > > *From*: Jay Vyas [mailto:jayunit...@gmail.com] > *Sent*: Tuesday, November 12, 2013 11:55 AM > *To*: user@bigtop.apache.org > *Subject*: East coast bigtop hackday/microtalks; Any interest? > > Hi folks. I

Re: East coast bigtop hackday/microtalks; Any interest?

2013-11-20 Thread Jay Vyas
feed to simulate a presence > effect ;) I will send the URL once we get closer to the event > > Cos > > On Sun, Nov 17, 2013 at 06:14PM, Roman Shaposhnik wrote: > > On Tue, Nov 12, 2013 at 8:55 AM, Jay Vyas wrote: > > > Hi folks. Is anyone interest in attending a big

Re: Getting Started Guide? (and some installation issues)

2013-11-21 Thread Jay Vyas
also, a good strategy with bigtop is to "start small"... Pick your use case, such as - I want to run bigtops smoke tests against my hadoop cluster for mahout, hive and pig to ensure that everything is working properly Or - I want to create custom rpm installs for my own hadoop and hive distr

Re: A first glance/reminder/hack at the BigPetStore pipeline

2013-12-03 Thread Jay Vyas
Thanks bruno ! I missed this :) . Yes ill add these in ! On Fri, Oct 25, 2013 at 4:50 AM, Bruno Mahé wrote: > On 10/08/2013 03:16 PM, Jay Vyas wrote: > >> Hi folks. >> >> Ive been hacking around on the big pet store idea. So far ive only got >> the templ

bigtop manual: how will the contents evolve?

2013-12-11 Thread Jay Vyas
ml Thanks and great work on creating a framework for code-based, human readable documentation :) -- Jay Vyas http://jayunit100.blogspot.com

Re: Bigtop - Oozie-setup script is missing

2014-01-10 Thread Jay Vyas
hmmm... How did you install it ? It's really easy to pick the bigtop ecosystem components you want and install them...An example of a quick way to "turn on" (i.e. yum install) the specific hadoop ecosystem components you need can be found here: https://github.com/apache/bigtop/blob/master/bi

does anyone else want a higher level interface to smokes

2014-01-21 Thread Jay Vyas
simpler error tracing (without having to dive into maven target/ dirs). I can work to make these wrappers as a start to this. but first want to get a feel if anyone else running the smokes would use a shell script wrapper preferentially. -- Jay Vyas http://jayunit100.blogspot.com

Re: does anyone else want a higher level interface to smokes

2014-01-27 Thread Jay Vyas
https://issues.apache.org/jira/browse/BIGTOP-1195 <-- moved here, added a prototypical patch On Sun, Jan 26, 2014 at 2:08 AM, Roman Shaposhnik wrote: > Hi! > > On Tue, Jan 21, 2014 at 10:58 AM, Jay Vyas wrote: > > Hi folks.. > > > > In the end for our smoke tes

Where does source actually come from ?

2014-02-21 Thread Jay Vyas
in not in the maven pom.xml , and references the cloudera/archives github repository for the source in the mk file. -- Jay Vyas http://jayunit100.blogspot.com

Re: Where does source actually come from ?

2014-02-21 Thread Jay Vyas
ere to download the > JARs for that component using Maven. > > > On Fri, Feb 21, 2014 at 2:14 PM, Jay Vyas wrote: > >> Hi bigtop ! >> >> I noticed that there are 2 locations of bigtop dependencies: >> >> 1) The pom.xml file >> >> 2) The expli

Re: does anyone else want a higher level interface to smokes

2014-02-23 Thread Jay Vyas
iginal Message - > > Hi! > > > > On Tue, Jan 21, 2014 at 10:58 AM, Jay Vyas wrote: > > > Hi folks.. > > > > > > In the end for our smoke tests, i have a shell script that wraps the > maven > > > stuff. > > > > > > Anyone else wa

ApacheCon ?

2014-03-03 Thread Jay Vyas
Any bigtop hackers goin to http://events.linuxfoundation.org/events/apachecon-north-america ? I'll be there. would like to organize a session. -- Jay Vyas http://jayunit100.blogspot.com

Re: ApacheCon ?

2014-03-03 Thread Jay Vyas
11:33 AM, Sean Mackrory > wrote: > > I know Roman, Cos and I are all presenting - it would be great to have a > > Bigtop meetup while we're there. > > > > > > On Mon, Mar 3, 2014 at 11:24 AM, Jay Vyas wrote: > >> > >> Any bigtop hackers goin

Re: ApacheCon ?

2014-03-03 Thread Jay Vyas
ay this living the bay > area!:-)), that'd be appreciated. > > >> On Mon, Mar 3, 2014 at 11:50 AM, Jay Vyas wrote: >> sure roman ! ill set it up. >> and red hat will sponsor as well / >> :) >> >> >> >> >>> On Mon, Mar

Re: ApacheCon ?

2014-03-03 Thread Jay Vyas
is gradle powered . So Id like to go through some of that code, and whiteboard how the whole stack will be looking inthe near future, so we all have an understanding of the new emerging bigtop architecture. On Mon, Mar 3, 2014 at 3:49 PM, Jay Vyas wrote: > Okay will do! We will put s

Re: ApacheCon ?

2014-03-06 Thread Jay Vyas
oman. > > > > > > (*) I'll totally repay by reviewing the stuff on my plate ;-) > > > > > > On Mon, Mar 3, 2014 at 11:33 AM, Sean Mackrory > wrote: > > > > I know Roman, Cos and I are all presenting - it would be great to > have a > &

Re: ApacheCon ?

2014-03-06 Thread Jay Vyas
days. up to you folks . i dont want to overwork you guys! On Thu, Mar 6, 2014 at 6:53 PM, Jay Vyas wrote: > Hi cos. > > > My plan is to find a room we can hang out in for 2 hours or so, and buy > pizza / food. > > Do you want to augment it and layer something more inter

Re: ApacheCon ?

2014-03-07 Thread Jay Vyas
me know if I can help with it. Let's try to finalize something say >> over the next week so we have time to make a noise about it;) >> >> Cos >> >> On Thu, Mar 06, 2014 at 09:37PM, Jay Vyas wrote: >> > Well, maybe we could do 2 meetups ? >> > >&

Re: Why is Apache BigTop named BigTop?

2014-03-21 Thread Jay Vyas
; Cos > > On Fri, Mar 21, 2014 at 07:38PM, Muhammad Asif Abbasi wrote: > > Hi, > > > > Can anybody help me figure out, why Apache BigTop is called BigTop? > > > > Appreciate all the help. > > > > Cheers, > -- Jay Vyas http://jayunit100.blogspot.com

Re: Why is Apache BigTop named BigTop?

2014-03-22 Thread Jay Vyas
well i think in actuality they'll still employ alot of people, but those people's value added to society will be much higher :) because rather than grepping log files, they'll be solving real world analytics problems.

Bigtop meetup ApacheCon

2014-03-25 Thread Jay Vyas
Hi bigtop ! Okay. So we finally have the meetup schedules solidified: April 7th AND 8th @ApacheCon, Denver, in the Westin - Teller Room, Please RSVP if possible in this thread so we can try to estimate how to handle food/beer etc. -- Jay Vyas http://jayunit100.blogspot.com

Re: Bigtop meetup ApacheCon

2014-04-04 Thread Jay Vyas
Apr 03, 2014 at 02:14AM, Konstantin Boudnik wrote: >>> I've posted the agenda to the wiki page. >>> >>> Please update it there if you want to add/changes anything. >>> >>> Cos >>> >>>> On Wed, Mar 26, 2014 at 01:19AM, Jay V

Re: Bigtop meetup ApacheCon

2014-04-05 Thread Jay Vyas
..(Bump).. Please RSVP on #asfbigtop for the meetups 7th 8th . Thanks! > On Apr 5, 2014, at 4:05 AM, Konstantin Boudnik wrote: > > RSVPs should be on twitter with #asfbigtop > -- > Regards, > Cos > >> On April 4, 2014 5:42:00 PM PDT, Jay Vyas wrote: >&

build server account --> *roman*

2014-04-08 Thread Jay Vyas
hi roman :) can you give me an account on the bigtop build server so that i can create a task to implement BIGTOP-1249 -- Jay Vyas http://jayunit100.blogspot.com

Should bigtop build mahout 2?

2014-04-09 Thread Jay Vyas
bigtop with the 1x version of mahout until they standardize to publishing 2x version of the jars. -- Jay Vyas http://jayunit100.blogspot.com

Re: Should bigtop build mahout 2?

2014-04-09 Thread Jay Vyas
>> mvn clean install -Dmahout.skip.distribution=false -DskipTests > -Dhadoop.version=$HADOOP_VERSION "$@" > > We should if there's any thing we're missing by using hadoop.version > instead of hadoop2.version, though... > > > On Wed, Apr 9, 2014 at 10:00 AM, Jay Vyas

Re: Bigtop meetup ApacheCon

2014-04-09 Thread Jay Vyas
ned about GridGain and OSV > as well as did some hacking around the latter technology. > > The ApacheCon was a lot of fun and interesting work! Was great to see > everyone! > Cos > >> On Wed, Mar 26, 2014 at 01:19AM, Jay Vyas wrote: >> Hi bigtop ! Okay. >>

Re: pig maven integration

2014-04-26 Thread Jay Vyas
This requires changes in script/code when there is a library upgrade. > > > -S > -- Jay Vyas http://jayunit100.blogspot.com

Can someone add me to Wiki editing?

2014-04-28 Thread Jay Vyas
Hi bigtop ! (maybe roman can do this, since we confirmed that you have privileges) Can someone add me as a wiki editor to bigtop ? Evidently bigtop admins can do it here: https://cwiki.apache.org/confluence/spaces/spacepermissions.action?key=BIGTOP -- Jay Vyas http://jayunit100.blogspot.com

Re: Can someone add me to Wiki editing?

2014-04-28 Thread Jay Vyas
et access. Something like this: > https://issues.apache.org/jira/browse/INFRA-7280 > > > On Tue, Apr 29, 2014 at 9:57 AM, Jay Vyas wrote: > >> Hi bigtop ! (maybe roman can do this, since we confirmed that you have >> privileges) >> >> Can someone add me as a wiki ed

MR Jobs failing in a vagrant deployment

2014-05-12 Thread Jay Vyas
pl.java:396) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) -- Jay Vyas http://jayunit100.blogspot.com

Re: MR Jobs failing in a vagrant deployment

2014-05-13 Thread Jay Vyas
the VM is using that or later version of hadoop, it might be worthwhile to > ensure property names are correct. > > Please see https://issues.apache.org/jira/browse/BIGTOP-1118 for details. > > Mark > > >> On Wed, May 7, 2014 at 8:48 PM, Jay Vyas wrote: >> Hi folk

Re: MR Jobs failing in a vagrant deployment

2014-05-17 Thread Jay Vyas
we can update puppet provisioner to be "Version aware" so that it warns or fails if you try to use it on a wrong version of hadoop ? On Tue, May 13, 2014 at 8:29 PM, Jay Vyas wrote: > No probably :) In the end I fixed it by exporting the right env variables > iirc > > On

BigTop Hackathon :::: Double Header ::: RSVP please !

2014-05-23 Thread Jay Vyas
ople.* If enough are interested, we will be able to two two days (friday the 6th and the 9th). Ping me back soon So i can start formalizing the arrangements ! -- Jay Vyas http://jayunit100.blogspot.com

Re: BigTop Hackathon :::: Double Header ::: RSVP please !

2014-05-23 Thread Jay Vyas
see as many of the contributors as possible! > And thanks to Jay (and RedHat for hosting us)! > > Andre, could you please post this on the meetup page as well? > Thanks, > Cos > > On Fri, May 23, 2014 at 06:20PM, Jay Vyas wrote: > > Hi folks. Ok we are all set ! >

Re: BigTop Hackathon :::: Double Header ::: RSVP please !

2014-05-23 Thread Jay Vyas
Red Hat offices, mountain view ! 444 Castro St, Mountain View, CA 9404 On Fri, May 23, 2014 at 8:17 PM, Giridharan Kesavan < gkesa...@hortonworks.com> wrote: > venue please? > > -giri > > > On Fri, May 23, 2014 at 4:36 PM, Jay Vyas wrote: > > > thanks anyone

puppet error (probably my fault)

2014-07-03 Thread jay vyas
quot;hadoop_namenode_host" variable --- probably this is a puppet variable scoping issue that I don't understand very well, or possibly a bug (but I doubt it since puppet recipes work just fine in vagrant). Also, maybe it could be a puppet versioning thing? Im using puppet 3.6.2 -- jay vyas

Re: puppet error (probably my fault)

2014-07-04 Thread jay vyas
onents, or some other system specific aspect of my config (I DONT think this could be a bug, given that so many people are succesfully using bigtop puppet recipes). On Thu, Jul 3, 2014 at 11:40 PM, jay vyas wrote: > Hi bigtop ! > > I've been playing with the puppet recipes,

Re: puppet error (probably my fault)

2014-07-04 Thread jay vyas
, jay vyas wrote: > I made some progress on this By adding > > notice("in common-hdfs $hadoop_namenode_host <<<< ") > file { >"/etc/hadoop/conf/core-site.xml" >content => template('hadoop/core-site.xml'), >require =&g

Re: New to Bigtop, where to start?

2014-07-16 Thread jay vyas
gt; sure where to start. I've cloned the github repo and used the startup.sh > script found in bigtop/bigtop-deploy/vm/vagrant-puppet to set up a virtual > cluster, but I am unsure how to apply this to physical machines. I'm also > not quite sure how to get hadoop and hdfs up an

  1   2   3   >