My curation of pending structured streaming PRs to review

2019-07-15 Thread Jungtaek Lim
Hi devs, As we make progress on some minor PRs on structured streaming, I'd like to remind about major PRs on SS area to get more chances to be reviewed. Please note that I only include existing PRs, so something still not discussed like queryable state is not included in the curation list.

RE: JDBC connector for DataSourceV2

2019-07-15 Thread Priyanka Gomatam
I would have thought one of the most important goals would be pushing down limits since V2 supports it. I am also interested in collaborating. Thanks! Priyanka Gomatam From: Shiv Prashant Sood Sent: Monday, July 15, 2019 10:22 AM To: Gabor Somogyi Cc: Xianyin Xin ; Ryan Blue ;

Creating external Druid table

2019-07-15 Thread Valeriy Trofimov
Hi All, How do you create an external Druid table via Spark? I know that you can do it like this: https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/using-druid/content/druid_anatomy_of_hive_to_druid.html But the issue is that Spark was built on Hive 1.2.1:

Re: JDBC connector for DataSourceV2

2019-07-15 Thread Shiv Prashant Sood
Agree. Let's use SPARK-24907 as the JIRA for this work. Thanks for resolving SPARK-28380 as dupe of this. Regards, Shiv On Mon, Jul 15, 2019 at 1:50 AM Gabor Somogyi wrote: > I've had a look

Re: Release Apache Spark 2.4.4 before 3.0.0

2019-07-15 Thread Dongjoon Hyun
Hi, Apache Spark PMC members. Can we cut Apache Spark 2.4.4 next Monday (22nd July)? Bests, Dongjoon. On Fri, Jul 12, 2019 at 3:18 PM Dongjoon Hyun wrote: > Thank you, Jacek. > > BTW, I added `@private` since we need PMC's help to make an Apache Spark > release. > > Can I get more feedbacks

Re: JDBC connector for DataSourceV2

2019-07-15 Thread Gabor Somogyi
I've had a look at the jiras and seems like the intention is the same (correct me if I'm wrong). I think one is enough and the rest can be closed with duplicate. We should keep multiple jiras only when the intention is different. BR, G On Mon, Jul 15, 2019 at 6:01 AM Xianyin Xin wrote: >