DataSourceV2 sync notes - 10 July 2019

2019-07-19 Thread Ryan Blue
Here are my notes from the last sync. If you’d like to be added to the invite or have topics, please let me know. *Attendees*: Ryan Blue Matt Cheah Yifei Huang Jose Torres Burak Yavuz Gengliang Wang Michael Artz Russel Spitzer *Topics*: - Existing PRs - V2 session catalog:

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

2019-07-19 Thread Hyukjin Kwon
That's a great explanation. Thanks I didn't know that. Josh, do you know who I should ping on this? On Fri, 19 Jul 2019, 16:52 Dongjoon Hyun, wrote: > Hi, Hyukjin. > > In short, there are two bots. And, the current situation happens when only > one bot with `dev/github_jira_sync.py` works. > >

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

2019-07-19 Thread Dongjoon Hyun
Hi, Hyukjin. In short, there are two bots. And, the current situation happens when only one bot with `dev/github_jira_sync.py` works. And, `dev/github_jira_sync.py` is irrelevant to the JIRA status change because it only use `add_remote_link` and `add_comment` API. I know only this bot (in

Spark and Oozie

2019-07-19 Thread Dennis Suhari
Dear experts, I am using Spark for processing data from HDFS (hadoop). These Spark application are data pipelines, data wrangling and machine learning applications. Thus Spark submits its job using YARN. This also works well. For scheduling I am now trying to use Apache Oozie, but I am