Spark Streaming Custom Receiver Anomaly

2018-02-20 Thread Thakrar, Jayesh
Hi All, I am trying to "test" a very simple custom receiver and am a little puzzled. Using Spark 2.2.0 shell on my laptop, I am running the code below. I was expecting the code to timeout since my timeout wait period is 1 ms and I have a sleep in the class that is much more (1200 ms). Is this n

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Weichen Xu
+1 On Wed, Feb 21, 2018 at 10:07 AM, Marcelo Vanzin wrote: > Done, thanks! > > On Tue, Feb 20, 2018 at 6:05 PM, Sameer Agarwal > wrote: > > Sure, please feel free to backport. > > > > On 20 February 2018 at 18:02, Marcelo Vanzin > wrote: > >> > >> Hey Sameer, > >> > >> Mind including https://g

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Marcelo Vanzin
Done, thanks! On Tue, Feb 20, 2018 at 6:05 PM, Sameer Agarwal wrote: > Sure, please feel free to backport. > > On 20 February 2018 at 18:02, Marcelo Vanzin wrote: >> >> Hey Sameer, >> >> Mind including https://github.com/apache/spark/pull/20643 >> (SPARK-23468) in the new RC? It's a minor bug s

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Sameer Agarwal
Sure, please feel free to backport. On 20 February 2018 at 18:02, Marcelo Vanzin wrote: > Hey Sameer, > > Mind including https://github.com/apache/spark/pull/20643 > (SPARK-23468) in the new RC? It's a minor bug since I've only hit it > with older shuffle services, but it's pretty safe. > > On

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Marcelo Vanzin
Hey Sameer, Mind including https://github.com/apache/spark/pull/20643 (SPARK-23468) in the new RC? It's a minor bug since I've only hit it with older shuffle services, but it's pretty safe. On Tue, Feb 20, 2018 at 5:58 PM, Sameer Agarwal wrote: > This RC has failed due to https://issues.apache.

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Sameer Agarwal
This RC has failed due to https://issues.apache.org/jira/browse/SPARK-23470. Now that the fix has been merged in 2.3 (thanks Marcelo!), I'll follow up with an RC5 soon. On 20 February 2018 at 16:49, Ryan Blue wrote: > +1 > > Build & tests look fine, checked signature and checksums for src tarbal

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Ryan Blue
+1 Build & tests look fine, checked signature and checksums for src tarball. On Tue, Feb 20, 2018 at 12:54 PM, Shixiong(Ryan) Zhu < shixi...@databricks.com> wrote: > I'm -1 because of the UI regression https://issues. > apache.org/jira/browse/SPARK-23470 : the All Jobs page may be too slow > and

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Shixiong(Ryan) Zhu
I'm -1 because of the UI regression https://issues.apache.org/jira /browse/SPARK-23470 : the All Jobs page may be too slow and cause "read timeout" when there are lots of jobs and stages. This is one of the most important pages because when it's broken, it's pretty hard to use Spark Web UI. On Tu

Re: How to change the attributes order in Apache SparkSQL `Project` operator ?

2018-02-20 Thread parana
Resolved using information from this post https://developer.ibm.com/code/2017/11/30/learn-extension-points-apache-spark-extend-spark-catalyst-optimizer/ -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ ---

Save the date: ApacheCon North America, September 24-27 in Montréal

2018-02-20 Thread Rich Bowen
Dear Apache Enthusiast, (You’re receiving this message because you’re subscribed to a user@ or dev@ list of one or more Apache Software Foundation projects.) We’re pleased to announce the upcoming ApacheCon [1] in Montréal, September 24-27. This event is all about you — the Apache project com

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Marco Gaido
+1 2018-02-20 12:30 GMT+01:00 Hyukjin Kwon : > +1 too > > 2018-02-20 14:41 GMT+09:00 Takuya UESHIN : > >> +1 >> >> >> On Tue, Feb 20, 2018 at 2:14 PM, Xingbo Jiang >> wrote: >> >>> +1 >>> >>> >>> Wenchen Fan 于2018年2月20日 周二下午1:09写道: >>> +1 On Tue, Feb 20, 2018 at 12:53 PM, Reynold

Re: [VOTE] Spark 2.3.0 (RC4)

2018-02-20 Thread Hyukjin Kwon
+1 too 2018-02-20 14:41 GMT+09:00 Takuya UESHIN : > +1 > > > On Tue, Feb 20, 2018 at 2:14 PM, Xingbo Jiang > wrote: > >> +1 >> >> >> Wenchen Fan 于2018年2月20日 周二下午1:09写道: >> >>> +1 >>> >>> On Tue, Feb 20, 2018 at 12:53 PM, Reynold Xin >>> wrote: >>> +1 On Feb 20, 2018, 5:51 PM +130