Re: [VOTE] Spark 2.3.0 (RC1)

2018-01-16 Thread Holden Karau
So looking at http://pgp.mit.edu/pks/lookup?op=vindex= 0xA1CEDBA8AD0C022A it seems like Sameer's key isn't in the Apache web of trust yet. This shouldn't block RC process but before we publish it's important to get the key in the Apache web of trust. On Tue, Jan 16, 2018 at 3:00 PM, Sameer

Re: [VOTE] Spark 2.3.0 (RC1)

2018-01-16 Thread Sameer Agarwal
Yes, I'll cut an RC2 as soon as the remaining blockers are resolved. In the meantime, please continue to report any other issues here. Here's a quick update on progress towards the next RC: - SPARK-22908 (KafkaContiniousSourceSuite) has been

Re: Distinct on Map data type -- SPARK-19893

2018-01-16 Thread Tejas Patil
There is a JIRA for making Map types orderable : https://issues.apache.org/jira/browse/SPARK-18134 Given that this is a non-trivial change, it will take time. On Sat, Jan 13, 2018 at 9:50 PM, ckhari4u wrote: > Wan, Thanks a lot,! I see the issue now. > > Do we have any

Re: [VOTE] Spark 2.3.0 (RC1)

2018-01-16 Thread Ted Yu
Is there going to be another RC ? With KafkaContinuousSourceSuite hanging, it is hard to get the rest of the tests going. Cheers On Sat, Jan 13, 2018 at 7:29 AM, Sean Owen wrote: > The signatures and licenses look OK. Except for the missing k8s package, > the contents look

Re: [build system] currently experiencing git timeouts when building

2018-01-16 Thread shane knapp
update: we're seeing about 3% of builds timing out, but today is (so far) a bad one: i've reached out to github about this and am waiting to hear back. $ get_timeouts.py 10 Timeouts by project: 17spark-branch-2.3-lint 10spark-branch-2.3-test-maven-hadoop-2.6 7

Re: [build system] yet another power outage at our colo

2018-01-16 Thread shane knapp
ok, we're back up and ready to build. sorry for the inconvenience. On Tue, Jan 16, 2018 at 9:59 AM, shane knapp wrote: > all non-UPS machines (read: all jenkins workers) temporarily lost power a > few minutes ago, and i will need to reconnect them to the master. > > this

[build system] yet another power outage at our colo

2018-01-16 Thread shane knapp
all non-UPS machines (read: all jenkins workers) temporarily lost power a few minutes ago, and i will need to reconnect them to the master. this means no builds for ~20 mins. i will also be installing a plugin for the spark-on-k8s builds (

Re: Whole-stage codegen and SparkPlan.newPredicate

2018-01-16 Thread Jacek Laskowski
Thanks for looking into it, Kazuaki! Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski Mastering Spark SQL https://bit.ly/mastering-spark-sql Spark Structured Streaming https://bit.ly/spark-structured-streaming Mastering Kafka Streams https://bit.ly/mastering-kafka-streams Follow