Re: [VOTE] Apache Spark 2.1.1 (RC2)

[Felix Cheung]

-1
sorry, found an issue with SparkR CRAN check.
Opened SPARK-20197 and working on fix.


From: holden.ka...@gmail.com  on behalf of Holden Karau 

Sent: Friday, March 31, 2017 6:25:20 PM
To: Xiao Li
Cc: Michael Armbrust; dev@spark.apache.org
Subject: Re: [VOTE] Apache Spark 2.1.1 (RC2)

-1 (non-binding)

Python packaging doesn't seem to have quite worked out (looking at PKG-INFO the 
description is "Description: ! missing pandoc do not upload to PyPI "), 
ideally it would be nice to have this as a version we upgrade to PyPi.
Building this on my own machine results in a longer description.

My guess is that whichever machine was used to package this is missing the 
pandoc executable (or possibly pypandoc library).

On Fri, Mar 31, 2017 at 3:40 PM, Xiao Li 
> wrote:
+1

Xiao

2017-03-30 16:09 GMT-07:00 Michael Armbrust 
>:
Please vote on releasing the following candidate as Apache Spark version 2.1.0. 
The vote is open until Sun, April 2nd, 2018 at 16:30 PST and passes if a 
majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.1
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see http://spark.apache.org/

The tag to be voted on is 
v2.1.1-rc2 
(02b165dcc2ee5245d1293a375a31660c9d4e1fa6)

List of JIRA tickets resolved can be found with this 
filter.

The release files, including signatures, digests, etc. can be found at:
http://home.apache.org/~pwendell/spark-releases/spark-2.1.1-rc2-bin/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1227/

The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-2.1.1-rc2-docs/


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an 
existing Spark workload and running on this release candidate, then reporting 
any regressions.

What should happen to JIRA tickets still targeting 2.1.1?

Committers should look at those and triage. Extremely important bug fixes, 
documentation, and API tweaks that impact compatibility should be worked on 
immediately. Everything else please retarget to 2.1.2 or 2.2.0.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless 
the bug in question is a regression from 2.1.0.

What happened to RC1?

There were issues with the release packaging and as a result was skipped.




--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau

Re: [VOTE] Apache Spark 2.1.1 (RC2)

[Kazuaki Ishizaki]

Thank you. Yes, it is not a regression. 2.1.0 would have this failure, 
too.

Regards,
Kazuaki Ishizaki



From:   Sean Owen 
To: Kazuaki Ishizaki/Japan/IBM@IBMJP, Michael Armbrust 

Cc: "dev@spark.apache.org" 
Date:   2017/04/02 18:18
Subject:Re: [VOTE] Apache Spark 2.1.1 (RC2)



That backport is fine, for another RC even in my opinion, but it's not a 
regression. It's a JDK bug really. 2.1.0 would have failed too.

On Sun, Apr 2, 2017 at 8:20 AM Kazuaki Ishizaki  
wrote:
-1 (non-binding)

I tested it on Ubuntu 16.04 and openjdk8 on ppc64le. I got several errors.
I expect that this backport (https://github.com/apache/spark/pull/17509) 
will be integrated into Spark 2.1.1.

Re: [VOTE] Apache Spark 2.1.1 (RC2)

[Sean Owen]

That backport is fine, for another RC even in my opinion, but it's not a
regression. It's a JDK bug really. 2.1.0 would have failed too.

On Sun, Apr 2, 2017 at 8:20 AM Kazuaki Ishizaki  wrote:

> -1 (non-binding)
>
> I tested it on Ubuntu 16.04 and openjdk8 on ppc64le. I got several errors.
> I expect that this backport (https://github.com/apache/spark/pull/17509)
> will be integrated into Spark 2.1.1.
>
>

Re: [VOTE] Apache Spark 2.1.1 (RC2)

[Kazuaki Ishizaki]

-1 (non-binding)

I tested it on Ubuntu 16.04 and openjdk8 on ppc64le. I got several errors.
I expect that this backport (https://github.com/apache/spark/pull/17509) 
will be integrated into Spark 2.1.1.


$ java -version
openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 
1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14)
OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
$ build/mvn -DskipTests -Phive -Phive-thriftserver -Pyarn -Phadoop-2.7 
package install
$ build/mvn -Phive -Phive-thriftserver -Pyarn -Phadoop-2.7 test -pl core
...
---
 T E S T S
---
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; 
support was removed in 8.0
Running org.apache.spark.memory.TaskMemoryManagerSuite
Tests run: 6, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.445 sec 
<<< FAILURE! - in org.apache.spark.memory.TaskMemoryManagerSuite
encodePageNumberAndOffsetOffHeap(org.apache.spark.memory.TaskMemoryManagerSuite)
 
 Time elapsed: 0.007 sec  <<< ERROR!
java.lang.IllegalArgumentException: requirement failed: No support for 
unaligned Unsafe. Set spark.memory.offHeap.enabled to false.
at 
org.apache.spark.memory.TaskMemoryManagerSuite.encodePageNumberAndOffsetOffHeap(TaskMemoryManagerSuite.java:48)

offHeapConfigurationBackwardsCompatibility(org.apache.spark.memory.TaskMemoryManagerSuite)
 
 Time elapsed: 0.013 sec  <<< ERROR!
java.lang.IllegalArgumentException: requirement failed: No support for 
unaligned Unsafe. Set spark.memory.offHeap.enabled to false.
at 
org.apache.spark.memory.TaskMemoryManagerSuite.offHeapConfigurationBackwardsCompatibility(TaskMemoryManagerSuite.java:138)

Running org.apache.spark.io.NioBufferedFileInputStreamSuite
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.029 sec 
- in org.apache.spark.io.NioBufferedFileInputStreamSuite
Running org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.708 sec 
<<< FAILURE! - in org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
testPeakMemoryUsed(org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite) 
 Time elapsed: 0.006 sec  <<< FAILURE!
java.lang.AssertionError: expected:<16648> but was:<16912>

Running org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
Tests run: 13, Failures: 0, Errors: 13, Skipped: 0, Time elapsed: 0.043 
sec <<< FAILURE! - in 
org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
failureToGrow(org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite) 
Time elapsed: 0.002 sec  <<< ERROR!
java.lang.IllegalArgumentException: requirement failed: No support for 
unaligned Unsafe. Set spark.memory.offHeap.enabled to false.
...
Tests run: 207, Failures: 7, Errors: 16, Skipped: 0

Kazuaki Ishizaki



From:   Michael Armbrust 
To: "dev@spark.apache.org" 
Date:   2017/03/31 08:10
Subject:[VOTE] Apache Spark 2.1.1 (RC2)



Please vote on releasing the following candidate as Apache Spark version 
2.1.0. The vote is open until Sun, April 2nd, 2018 at 16:30 PST and passes 
if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.1
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see http://spark.apache.org/

The tag to be voted on is v2.1.1-rc2 (
02b165dcc2ee5245d1293a375a31660c9d4e1fa6)

List of JIRA tickets resolved can be found with this filter.

The release files, including signatures, digests, etc. can be found at:
http://home.apache.org/~pwendell/spark-releases/spark-2.1.1-rc2-bin/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1227/

The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-2.1.1-rc2-docs/


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an 
existing Spark workload and running on this release candidate, then 
reporting any regressions.

What should happen to JIRA tickets still targeting 2.1.1?

Committers should look at those and triage. Extremely important bug fixes, 
documentation, and API tweaks that impact compatibility should be worked 
on immediately. Everything else please retarget to 2.1.2 or 2.2.0.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release 
unless the bug in question is a regression from 2.1.0.

What happened to RC1?

There were issues with the release packaging and as a result was skipped.