If so, probably we need to add the SQL dialects switching support for
SparkSQLCLI, as Fei suggested. What do you think the priority for this?
-Original Message-
From: Cheng Lian [mailto:lian.cs@gmail.com]
Sent: Friday, August 15, 2014 1:57 PM
To: Cheng, Hao
Cc: scwf;
On Sun, Aug 3, 2014 at 4:35 PM, Nicholas Chammas nicholas.cham...@gmail.com
wrote:
Include the commit hash in the tests have started/completed messages, so
that it's clear what code exactly is/has been tested for each test cycle.
This is now captured in this JIRA issue
I am still a bit confused that why this issue did not show up in 0.9...at
that time there was no spark-submit and the context was constructed with
low level calls...
Kryo register for ALS was always in my application code..
Was this bug introduced in 1.0 or it was always there ?
On Aug 14, 2014
Hi All,
I noticed that all PR tests run overnight had failed due to timeouts. The
patch that updates the netty shuffle I believe somehow inflated to the
build time significantly. That patch had been tested, but one change was
made before it was merged that was not tested.
I've reverted the patch
Thanks for your response. I think I misinterpreted the
stability/compatibility guarantee with 1.0 release. It seems like the
compatibility is only at the API level.
This is interesting because it means any system/product that is built on top
of Spark and uses Spark with a long-running
Hi Xiangrui,
I wasn't setting spark.driver.memory. I'll try that and report back.
In terms of this running on the cluster, my assumption was that calling foreach
on an array(I converted samples using toArray) would mean counts is propagated
locally. The object would then be serialized to
Also I think Jenkins doesn't post build timeouts to github. Is there anyway
we can fix that ?
On Aug 15, 2014 9:04 AM, Patrick Wendell pwend...@gmail.com wrote:
Hi All,
I noticed that all PR tests run overnight had failed due to timeouts. The
patch that updates the netty shuffle I believe
Shivaram,
Can you point us to an example of that happening? The Jenkins console
output, that is.
Nick
On Fri, Aug 15, 2014 at 2:28 PM, Shivaram Venkataraman
shiva...@eecs.berkeley.edu wrote:
Also I think Jenkins doesn't post build timeouts to github. Is there anyway
we can fix that ?
On
Setting spark.driver.memory has no effect. It's still hanging trying to
compute result.count when I'm sampling greater than 35% regardless of what
value of spark.driver.memory I'm setting.
Here's my settings:
export SPARK_JAVA_OPTS=-Xms5g -Xmx10g -XX:MaxPermSize=10g
export SPARK_MEM=10g
in
Hey Nicholas,
Yeah so Jenkins has it's own timeout mechanism and it will just kill the
entire build after 120 minutes. But since run-tests is sitting in the
middle of the tests, it can't actually post a failure message.
I think run-tests-jenkins should just wrap the call to run-tests in a call
So 2 hours is a hard cap on how long a build can run. Okie doke.
Perhaps then I'll wrap the run-tests step as you suggest and limit it to
100 minutes or something, and cleanly report if it times out.
Sound good?
On Fri, Aug 15, 2014 at 4:43 PM, Patrick Wendell pwend...@gmail.com wrote:
Hey
Yeah I was thinking something like that. Basically we should just have a
variable for the timeout and I can make sure it's under the configured
Jenkins time.
On Fri, Aug 15, 2014 at 1:55 PM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
So 2 hours is a hard cap on how long a build can
I've opened SPARK-3051 (https://issues.apache.org/jira/browse/SPARK-3051)
based on this thread.
Neil
On Thu, Jul 24, 2014 at 10:30 PM, Neil Ferguson nfergu...@gmail.com wrote:
That would work well for me! Do you think it would be necessary to specify
which accumulators should be available in
13 matches
Mail list logo
