Re: Increase Timeout or optimize Spark UT?

+1 for any efforts to recover Jenkins! Thank you for the direction. Bests, Dongjoon. From: Reynold Xin Date: Sunday, August 20, 2017 at 5:53 PM To: Dong Joon Hyun Cc: "dev@spark.apache.org" Subject: Re: Increase Timeout or

Re: Increase Timeout or optimize Spark UT?

It seems like it's time to look into how to cut down some of the test runtimes. Test runtimes will slowly go up given the way development happens. 3 hr is already a very long time for tests to run. On Sun, Aug 20, 2017 at 5:45 PM, Dong Joon Hyun wrote: > Hi, All. > > > >

Increase Timeout or optimize Spark UT?

Hi, All. Recently, Apache Spark master branch test (SBT with hadoop-2.7 / 2.6) has been hitting the build timeout. Please see the build time trend. https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7/buildTimeTrend All recent 22

Question on how to get appended data from structured streaming

Hello, I am new to Spark. It would be appreciated if anyone could help me understand how to get appended data from structured streaming. According to the document , data stream could be treated as new