Hey Kevin,
If you are upgrading from 1.0.X to 1.1.X checkout the upgrade notes
here [1] - it could be that default changes caused a regression for
your workload. Do you still see a regression if you restore the
configuration changes?
It's great to hear specifically about issues like this, so plea
+0 (non-binding)
Compiled Spark, recompiled and ran application with 1.1.1 RC1 with Yarn,
plain-vanilla Hadoop 2.3.0. No regressions.
However, 12% to 22% increase in run time relative to 1.0.0 release. (No
other environment or configuration changes.) Would have recommended +1
were it not f
Andrew,
I put up 1.1.1 branch and I am getting shuffle failures while doing flatMap
followed by groupBy...My cluster memory is less than the memory I need and
therefore flatMap does around 400 GB of shuffle...memory is around 120 GB...
14/11/13 23:10:49 WARN TaskSetManager: Lost task 22.1 in stag
This seems like a legitimate blocker. We will cut another RC to include the
revert.
2014-11-16 17:29 GMT-08:00 Kousuke Saruta :
> Now I've finished to revert for SPARK-4434 and opened PR.
>
>
> (2014/11/16 17:08), Josh Rosen wrote:
>
>> -1
>>
>> I found a potential regression in 1.1.1 related to
Now I've finished to revert for SPARK-4434 and opened PR.
(2014/11/16 17:08), Josh Rosen wrote:
-1
I found a potential regression in 1.1.1 related to spark-submit and cluster
deploy mode: https://issues.apache.org/jira/browse/SPARK-4434
I think that this is worth fixing.
On Fri, Nov 14, 2014
-1
I found a potential regression in 1.1.1 related to spark-submit and cluster
deploy mode: https://issues.apache.org/jira/browse/SPARK-4434
I think that this is worth fixing.
On Fri, Nov 14, 2014 at 7:28 PM, Cheng Lian wrote:
> +1
>
> Tested HiveThriftServer2 against Hive 0.12.0 on Mac OS X.
+1
Tested HiveThriftServer2 against Hive 0.12.0 on Mac OS X. Known issues
are fixed. Hive version inspection works as expected.
On 11/15/14 8:25 AM, Zach Fry wrote:
+0
I expect to start testing on Monday but won't have enough results to change
my vote from +0
until Monday night or Tuesday mo
+0
I expect to start testing on Monday but won't have enough results to change
my vote from +0
until Monday night or Tuesday morning.
Thanks,
Zach
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-1-1-RC1-tp9311p9370.html
S
+1
Tested on Mac OS X, and verified that sort-based shuffle bug is fixed.
Matei
> On Nov 14, 2014, at 10:45 AM, Andrew Or wrote:
>
> Hi all, since the vote ends on a Sunday, please let me know if you would
> like to extend the deadline to allow more time for testing.
>
> 2014-11-13 12:10 GMT-
Hi all, since the vote ends on a Sunday, please let me know if you would
like to extend the deadline to allow more time for testing.
2014-11-13 12:10 GMT-08:00 Sean Owen :
> Ah right. This is because I'm running Java 8. This was fixed in
> SPARK-3329 (
> https://github.com/apache/spark/commit/2b7
Ah right. This is because I'm running Java 8. This was fixed in
SPARK-3329
(https://github.com/apache/spark/commit/2b7ab814f9bde65ebc57ebd04386e56c97f06f4a#diff-7bfd8d7c8cbb02aa0023e4c3497ee832).
Consider back-porting it if other reasons arise, but this is specific
to tests and to Java 8.
On Thu,
Yeah, this seems to be somewhat environment specific too. The same test has
been passing here for a while:
https://amplab.cs.berkeley.edu/jenkins/job/Spark-1.1-Maven-pre-YARN/hadoop.version=1.0.4,label=centos/lastBuild/consoleFull
2014-11-13 11:26 GMT-08:00 Michael Armbrust :
> Hey Sean,
>
> Than
Hey Sean,
Thanks for pointing this out. Looks like a bad test where we should be
doing Set comparison instead of Array.
Michael
On Thu, Nov 13, 2014 at 2:05 AM, Sean Owen wrote:
> LICENSE and NOTICE are fine. Signature and checksum is fine. I
> unzipped and built the plain source distribution
LICENSE and NOTICE are fine. Signature and checksum is fine. I
unzipped and built the plain source distribution, which built.
However I am seeing a consistent test failure with "mvn -DskipTests
clean package; mvn test". In the Hive module:
- SET commands semantics for a HiveContext *** FAILED ***
+1
1. Compiled OSX 10.10 (Yosemite) mvn -Pyarn -Phadoop-2.4
-Dhadoop.version=2.4.0 -DskipTests clean package 10:49 min
2. Tested pyspark, mlib
2.1. statistics OK
2.2. Linear/Ridge/Laso Regression OK
2.3. Decision Tree, Naive Bayes OK
2.4. KMeans OK
2.5. rdd operations OK
2.6. recommendation OK
2.7.
I will start the vote with a +1
2014-11-12 20:34 GMT-08:00 Andrew Or :
> Please vote on releasing the following candidate as Apache Spark version 1
> .1.1.
>
> This release fixes a number of bugs in Spark 1.1.0. Some of the notable
> ones are
> - [SPARK-3426] Sort-based shuffle compression settin
16 matches
Mail list logo
