Marcelo
Thanks for the comments. All my requirements are from our work over last
year in yarn-cluster mode. So I am biased on the yarn side.
It's true some of the task might be able accomplished with a separate
yarn API call, the API just does not same to be that nature any more if
Spark SQL just load the query result as a new source (via JDBC), so DO NOT
confused with the Spark SQL tables. They are totally independent database
systems.
From: Yi Zhang [mailto:zhangy...@yahoo.com.INVALID]
Sent: Friday, May 15, 2015 1:59 PM
To: Cheng, Hao; Dev
Subject: Re: Does Spark SQL
Hi,
May I know the tentative release dates for spark 1.3.2?
rgds
--
Niranda
Jenkins build against hadoop 2.4 has been unstable recently:
https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-Maven-with-YARN/HADOOP_PROFILE=hadoop-2.4,label=centos/
I haven't found the test which hung / failed in recent Jenkins builds.
But PR builder has several green builds
Hi Niranda,
Maintenance releases are not done on a predetermined schedule but
instead according to which fixes show up and their severity. Since we
just did a 1.3.1 release I'm not sure I see 1.3.2 on the immediate
horizon.
However, the maintenance releases are simply builds at the head of the
From
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32831/consoleFull
:
[info] Building Spark with these arguments: -Pyarn -Phadoop-2.3
-Dhadoop.version=2.3.0 -Pkinesis-asl -Phive -Phive-thriftserver
Should PR builder cover hadoop 2.4 as well ?
Thanks
On Fri, May 15,
If there is no further feedback on this I will ask ASF Infra to add
the new fields Out of Scope and Inactive.
- Patrick
On Tue, May 12, 2015 at 9:02 AM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
I tend to find that any large project has a lot of walking dead JIRAs, and
pretending they
Funny thing, since I asked this question in a PR a few minutes ago...
Ignoring the rotation suggestion for a second, can the PR builder at least
cover hadoop 2.2? That's the actual version used to create the official
Spark artifacts for maven, and the oldest version Spark supports for YARN..
The PR builder currently builds against Hadoop 2.3.
- Patrick
On Fri, May 15, 2015 at 11:40 AM, Marcelo Vanzin van...@cloudera.com
wrote:
Funny thing, since I asked this question in a PR a few minutes ago...
Ignoring the rotation suggestion for a second, can the PR builder at least
cover
bq. would be prohibitive to build all configurations for every push
Agreed.
Can PR builder rotate testing against hadoop 2.3, 2.4, 2.6 and 2.7 (each
test run still uses one hadoop profile) ?
This way we would have some coverage for each of the major hadoop releases.
Cheers
On Fri, May 15,
Sorry premature send:
The PR builder currently builds against Hadoop 2.3
https://github.com/apache/spark/blob/master/dev/run-tests#L54
We can set this to whatever we want. 2.2 might make sense since it's the
default in our published artifacts.
- Patrick
On Fri, May 15, 2015 at 11:53 AM,
The PR builder seems to be building against Hadoop 2.3. In the log for the
most recent successful build (
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32805/consoleFull
) I see:
=
Building Spark
You all are looking only at the pull request builder. It just does one
build to sanity-check a pull request, since that already takes 2 hours and
would be prohibitive to build all configurations for every push. There is a
different set of Jenkins jobs that periodically tests master against a lot
Hi Chester,
Writing a design / requirements doc sounds great. One comment though:
On Thu, May 14, 2015 at 11:18 PM, Chester At Work ches...@alpinenow.com
wrote:
For #5 yes, it's about the command line args. These are args are
the input for the spark jobs. Seems a bit too much to create
14 matches
Mail list logo
