Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

2019-01-23 Thread Takeshi Yamamuro
Thanks, all. I'll start a new vote as rc2 after the two issues above resolved. Best, Takeshi On Thu, Jan 24, 2019 at 7:59 AM Xiao Li wrote: > -1 > > https://issues.apache.org/jira/browse/SPARK-26709 is another blocker > ticket that returns incorrect results. > > > Marcelo Vanzin

Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

2019-01-23 Thread Xiao Li
-1 https://issues.apache.org/jira/browse/SPARK-26709 is another blocker ticket that returns incorrect results. Marcelo Vanzin 于2019年1月23日周三 下午12:01写道: > -1 too. > > I just upgraded https://issues.apache.org/jira/browse/SPARK-26682 to > blocker. It's a small fix and we should make it in 2.3.3.

Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

2019-01-23 Thread Marcelo Vanzin
-1 too. I just upgraded https://issues.apache.org/jira/browse/SPARK-26682 to blocker. It's a small fix and we should make it in 2.3.3. On Thu, Jan 17, 2019 at 6:49 PM Takeshi Yamamuro wrote: > > Please vote on releasing the following candidate as Apache Spark version > 2.3.3. > > The vote is

Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

2019-01-23 Thread DB Tsai
-1 Agreed with Anton that this bug will potentially corrupt the data silently. As he is ready to submit a PR, I'll suggest to wait to include the fix. Thanks! Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0x5CED8B896A6BDFA0

Re: Removing old HiveMetastore(0.12~0.14) from Spark 3.0.0?

2019-01-23 Thread Reynold Xin
It is not even an old “cluster”. It is a central metastore shares by multiple clusters. On Wed, Jan 23, 2019 at 10:04 AM Dongjoon Hyun wrote: > Got it. Thank you for sharing that, Reynold. > > So, you mean they will use `Apache Spark 3.0.0` on the old clusters with > Hive 0.x, right? > > If

Re: Removing old HiveMetastore(0.12~0.14) from Spark 3.0.0?

2019-01-23 Thread Dongjoon Hyun
Got it. Thank you for sharing that, Reynold. So, you mean they will use `Apache Spark 3.0.0` on the old clusters with Hive 0.x, right? If that happens actually, no problem to keep them. Bests, Dongjoon. On Tue, Jan 22, 2019 at 11:49 PM Xiao Li wrote: > Based on my experience in development

Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

2019-01-23 Thread Anton Okolnychyi
It is a correctness bug. I have updated the description with an example. It has been there for a while, so I am not sure about the priority. ср, 23 янв. 2019 г. в 14:48, Sean Owen : > I'm not clear if it's a correctness bug from that description, and if > it's not a regression, no it does not

Custom Window Function

2019-01-23 Thread Georg Heiler
Hi, I want to write custom window functions in spark which are also optimisable for catalyst. Can you provide some hints where to start? Also posting to DEVLIST as I believe this is a rather exotic topic. Best, Georg

Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

2019-01-23 Thread Sean Owen
I'm not clear if it's a correctness bug from that description, and if it's not a regression, no it does not need to go into 2.3.3. If it's a real bug, sure it can be merged to 2.3.x. On Wed, Jan 23, 2019 at 7:54 AM Anton Okolnychyi wrote: > > Recently, I came across this bug: >

Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

2019-01-23 Thread Anton Okolnychyi
Recently, I came across this bug: https://issues.apache.org/jira/browse/SPARK-26706. It seems appropriate to include it in 2.3.3, doesn't it? Thanks, Anton ср, 23 янв. 2019 г. в 13:08, Takeshi Yamamuro : > Thanks for the check, Felix! > > Yea, I'll wait for the new test report. > But, it never