Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 On Sun, Sep 25, 2016 at 3:26 PM, Herman van Hövell tot Westerflier < hvanhov...@databricks.com> wrote: > +1 (non-binding) > > On Sun, Sep 25, 2016 at 2:05 PM, Ricardo Almeida < > ricardo.alme...@actnowib.com> wrote: > >> +1 (non-binding) >> >> Built and tested on >> - Ubuntu 16.04 / OpenJDK

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 (non-binding) On Sat, Sep 24, 2016 at 3:08 PM, Reynold Xin wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.0.1. The vote is open until Tue, Sep 27, 2016 at 15:30 PDT and passes if > a majority of at least 3+1 PMC votes are cast. > >

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 (non-binding) Regards, Vaquar khan On 25 Sep 2016 20:41, "Kousuke Saruta" wrote: > +1 (non-binding) > > On 2016年09月26日 07:26, Herman van Hövell tot Westerflier wrote: > > +1 (non-binding) > > On Sun, Sep 25, 2016 at 2:05 PM, Ricardo Almeida < >

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 (non-binding) On 2016年09月26日 07:26, Herman van Hövell tot Westerflier wrote: +1 (non-binding) On Sun, Sep 25, 2016 at 2:05 PM, Ricardo Almeida > wrote: +1 (non-binding) Built and tested on - Ubuntu 16.04 /

Re: ArrayType support in Spark SQL

Continuing to dig, I encountered: https://github.com/apache/spark/blob/master/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralExpressionSuite.scala#L125 // TODO(davies): add tests for ArrayType, MapType and StructType I guess others have thought of this already,

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 (non-binding) On Sun, Sep 25, 2016 at 2:05 PM, Ricardo Almeida < ricardo.alme...@actnowib.com> wrote: > +1 (non-binding) > > Built and tested on > - Ubuntu 16.04 / OpenJDK 1.8.0_91 > - CentOS / Oracle Java 1.7.0_55 > (-Phadoop-2.7 -Dhadoop.version=2.7.3 -Phive -Phive-thriftserver -Pyarn) > >

ArrayType support in Spark SQL

It seems that `functions.lit` doesn't support ArrayTypes. To reproduce: org.apache.spark.sql.functions.lit(2 :: 1 :: Nil) java.lang.RuntimeException: Unsupported literal type class scala.collection.immutable.$colon$colon List(2, 1) at

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Spark's branch-2.0 is a maintenance branch, effectively meaning that only bug-fixes will be added to it. There are other maintenance branches (such as branch-1.6) that are also receiving bug-fixes in theory, but not so much in fact as maintenance branches get older. The major and minor version

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 (non-binding) Built and tested on - Ubuntu 16.04 / OpenJDK 1.8.0_91 - CentOS / Oracle Java 1.7.0_55 (-Phadoop-2.7 -Dhadoop.version=2.7.3 -Phive -Phive-thriftserver -Pyarn) On 25 September 2016 at 22:35, Matei Zaharia wrote: > +1 > > Matei > > On Sep 25, 2016, at

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 Matei > On Sep 25, 2016, at 1:25 PM, Josh Rosen wrote: > > +1 > > On Sun, Sep 25, 2016 at 1:16 PM Yin Huai > wrote: > +1 > > On Sun, Sep 25, 2016 at 11:40 AM, Dongjoon Hyun

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 On Sun, Sep 25, 2016 at 1:16 PM Yin Huai wrote: > +1 > > On Sun, Sep 25, 2016 at 11:40 AM, Dongjoon Hyun > wrote: > >> +1 (non binding) >> >> RC3 is compiled and tested on the following two systems, too. All tests >> passed. >> >> * CentOS 7.2 /

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 On Sun, Sep 25, 2016 at 11:40 AM, Dongjoon Hyun wrote: > +1 (non binding) > > RC3 is compiled and tested on the following two systems, too. All tests > passed. > > * CentOS 7.2 / Oracle JDK 1.8.0_77 / R 3.3.1 >with -Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 (non binding) RC3 is compiled and tested on the following two systems, too. All tests passed. * CentOS 7.2 / Oracle JDK 1.8.0_77 / R 3.3.1 with -Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver -Dsparkr * CentOS 7.2 / Open JDK 1.8.0_102 with -Pyarn -Phadoop-2.7 -Pkinesis-asl

Master, branches and versioning

(Renaming thread to keep it separate from RC vote) If you're asking why there's a version 2.0.2 in JIRA, it's because we have to have that entity in order to target anything to version 2.0.2. 2.2.0 exists as well as version labels in JIRA. None are marked as 'released' because there is no such

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 Ship it! Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Sun, Sep 25, 2016 at 12:08 AM, Reynold Xin wrote: > Please vote on releasing

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Hi Sean, I remember a similar discussion about the releases in Spark and I must admit it again -- I simply don't get it. I seem to not have paid enough attention to details to appreciate it. I apologize for asking the very same questions again and again. Sorry. Re the next release, I was

Re: [VOTE] Release Apache Spark 2.0.1 (RC3)

+1 binding. Same as for RC2 -- it all checks out, from license to sigs to compile and test. We have no issues of any kind targeted for 2.0.1. I do have one test failure that I have seen with some regularity in Ubuntu but not on any Jenkins machines. One of the YARN tests will just kill all my ssh

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Master is implicitly 2.1.x right now. When branch-2.1 is cut, master becomes the de facto 2.2.x branch. It's not true that the next release is 2.0.2. You can see the master version: https://github.com/apache/spark/blob/master/pom.xml#L29 On Sun, Sep 25, 2016 at 12:30 PM, Jacek Laskowski

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Hi Sean, So, another question would be when is the change going to be released then? What's the version for the master? The next release's 2.0.2 so it's not for mesos profile either :( Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

It's a change to the structure of the project, and probably not appropriate for a maintenance release. 2.0.1 core would then no longer contain Mesos code while 2.0.0 did. On Sun, Sep 25, 2016 at 12:26 PM, Jacek Laskowski wrote: > Hi Sean, > > Sure, but then the question is why

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Hi Sean, Sure, but then the question is why it's not a part of 2.0.1? I thought it was considered ready for prime time and so should be shipped in 2.0.1. Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

It was added to the master branch, and this is a release from the 2.0.x branch. On Sun, Sep 25, 2016 at 12:12 PM, Jacek Laskowski wrote: > Hi, > > That's even more interesting. How's so since the profile got added a > week ago or later and RC2 was cut two/three days ago? Anyone

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Hi, That's even more interesting. How's so since the profile got added a week ago or later and RC2 was cut two/three days ago? Anyone know? Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me at