Re: Proposal for JIRA and Pull Request Policy

2014-02-11 Thread Tom Graves
+1 This is a bit aside, but are we also getting jenkins machine in the apache domain? Tom On Friday, February 7, 2014 1:54 AM, Kostas Sakellis wrote: +1 On Thursday, February 6, 2014, Sandy Ryza wrote: > +1 > > > On Thu, Feb 6, 2014 at 4:10 PM, Mridul Muralidharan > > >wrote: > > > +

Re: [VOTE] Graduation of Apache Spark from the Incubator

2014-02-11 Thread Tom Graves
+1. Tom On Tuesday, February 11, 2014 10:13 AM, Andrew Psaltis wrote: +1 On Tue, Feb 11, 2014 at 8:50 AM, Giri Iyengar wrote: > +1 > > -- > GIRI IYENGAR, CTO > VELOS.IO > Simple. Powerful. Predictions. > > 440 NINTH AVE, 11TH FLOOR NEW YORK CITY, NY 10001 > O: 917.525.2466x104  M: 914

Re: [DISCUSS] Graduating as a TLP

2014-01-23 Thread Tom Graves
+1 Tom On Thursday, January 23, 2014 6:14 PM, Reynold Xin wrote: +1 supporting Matei as the VP. On Thu, Jan 23, 2014 at 4:11 PM, Chris Mattmann wrote: > +1 from me. > > I'll throw Matei's name into the hat for VP. He's done a great job > and has stood out to me with his report filing an

Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc4) [new thread]

2014-01-23 Thread Tom Graves
+1.   verified sigs/hashes, NOTICE, LICENSE Built from source on redhat linux using maven. Ran some basic tests on secure Hadoop YARN 0.23 and 2.2 clusters in both standalone and client modes. Tom On Wednesday, January 22, 2014 12:28 PM, Khanderao kand wrote: +1 However please fix a mino

spork?

2014-01-22 Thread Tom Graves
Hey everyone, At one point I heard of pig on Spark (Spork).  The only thing I could find on it was https://github.com/dvryaboy/pig and it doesn't look like there has been any work on it in quite a while.   Does anyone know the current status of that or has tried it? Thanks, Tom

Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc4) [new thread]

2014-01-22 Thread Tom Graves
It looks like the latest round of changes took out spark.conf.  Are there plans to add this back in (jira)? Tom On Wednesday, January 22, 2014 3:46 AM, Henry Saputra wrote: Would love to hear from Mridul to verify the fixes for problems he saw are in. On Tuesday, January 21, 2014, Patric

spark compatibility between releases?

2014-01-17 Thread Tom Graves
Hey everyone, What is the story as far as Spark compatibility between releases?  Are we saying recompile needed until we get to 1.X type releases?   It might be a good idea for us to document our compatibility strategy going forward.  I had someone say they built against master and tried to ru

Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc1)

2014-01-17 Thread Tom Graves
+1.   verified sigs/hashes, NOTICE, LICENSE Built from source on redhat linux using maven. Ran some tests on secure Hadoop YARN 0.23 and 2.2 clusters in both standalone and client modes. Tom On Wednesday, January 15, 2014 7:49 PM, Patrick Wendell wrote: Please vote on releasing the follow

Re: About Spark job web ui persist(JIRA-969)

2014-01-13 Thread Tom Graves
This way Spark's history information is persisted independent of cluster framework, and cluster frameworks can import the history when/as needed. Hope this helps. Regards, pillis On Thu, Jan 9, 2014 at 6:13 AM, Tom Graves wrote: > Note that it looks like we are planning on adding suppo

Re: About Spark job web ui persist(JIRA-969)

2014-01-09 Thread Tom Graves
ce's JobHistoryServer.  It will > > eventually have the ability to store application-specific data. > > > > -Sandy > > > > > > On Tue, Jan 7, 2014 at 2:51 PM, Tom Graves wrote: > > > >> I don't think you want to save the html/xml files. I would

Re: About Spark job web ui persist(JIRA-969)

2014-01-07 Thread Tom Graves
I don't think you want to save the html/xml files. I would rather see the info saved into a history file in like a json format that could then be re-read and the web ui display the info, hopefully without much change to the UI parts.   For instance perhaps the history server could read the file a

Re: sbt to run a single test

2014-01-06 Thread Tom Graves
/sbt "test-only org.apache.spark.BroadcastSuite" > >I.e. As single argument. Hope that helps.  > > > >On Monday, January 6, 2014, Tom Graves wrote: > >Hey all, >> >>I haven't used sbt for spark much, mostly maven. I have been using it now >>

sbt to run a single test

2014-01-06 Thread Tom Graves
Hey all, I haven't used sbt for spark much, mostly maven. I have been using it now recently and want to run just a single test.  Is there a way to do it? I tried sbt test-only it didn't seem to work but perhaps my syntax was wrong.   SPARK_HADOOP_VERSION=0.23.10  SPARK_YARN=true sbt/sbt test-only

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-12 Thread Tom Graves
+1.  Built spark on yarn for both hadoop 0.23 and hadoop 2.2.0 on redhat linux using maven.  Ran some tests on both a secure Hadoop 0.23 cluster and a secure Hadoop 2.2.0 cluster.  Verified signatures and md5. Tom On Tuesday, December 10, 2013 6:49 PM, Patrick Wendell wrote: Please vote

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-12 Thread Tom Graves
I re-downloaded the source tarball and it works now. Tom On Wednesday, December 11, 2013 6:27 PM, Patrick Wendell wrote: I also talked to a few people who got corrupted binaries when downloading from the people.apache HTTP. In that case the checksum failed but if they re-downloaded it worke

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-11 Thread Tom Graves
Hey folks, I'm trying to verify the signature on the rc4 but am getting a BAD signature, is it working for others?  Perhaps I messed up the import of the asc file. $ gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz gpg: Signature made Tue 10 Dec 2013 10:53:15 PM UTC using R

Re: [ANNOUNCE] Welcoming two new Spark committers: Tom Graves and Prashant Sharma

2013-11-15 Thread Tom Graves
t;> On Thu, Nov 14, 2013 at 12:21 AM, karthik tunga > >wrote: >> >> > Congrats Tom and Prashant :) >> > >> > Cheers, >> > Karthik >> > On Nov 13, 2013 7:37 PM, "Matei Zaharia" >> wrote: >> > >> > > Hi folk

logging in the unit tests

2013-09-23 Thread Tom Graves
Does anyone know how to turn on the logging in the unit tests?  I'm using maven and all I see in the txt output files are the test names.  I want to see the normal Spark logging for stdout/stderr. Thanks, Tom

Moving to apache incubator git?

2013-09-20 Thread Tom Graves
Hey all, Sorry if I missed it on here.  I'm curious if there is a plan on when we are moving to the apache incubator git repo?  When should pull requests start going there?  The master branch seems a bit behind from the mesos/spark branch. Thanks, Tom

Re: Question about Spark on Yarn design

2013-08-23 Thread Tom Graves
We also want to eventually make spark on yarn be able to handle interactive and be more like the standalone deploy where it is more long lived.  The first step was to it working for the batch like jobs.  So we plan on working on that later this year.  If you are interested in working on it that