[jira] [Resolved] (GOBBLIN-432) Share the DataSource used by the MySQL state stores

2018-03-21 Thread Hung Tran (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hung Tran resolved GOBBLIN-432. --- Resolution: Fixed Fix Version/s: 0.13.0 Issue resolved by pull request #2311

[jira] [Updated] (GOBBLIN-433) Gobblin tries to query schema registry for non existing Kafka partitions

2018-03-21 Thread Tamas Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Nemeth updated GOBBLIN-433: - Summary: Gobblin tries to query schema registry for non existing Kafka partitions (was: Try to

[jira] [Created] (GOBBLIN-434) Salesforce connector support refresh token grant

2018-03-21 Thread HAO JI WU (JIRA)
HAO JI WU created GOBBLIN-434: - Summary: Salesforce connector support refresh token grant Key: GOBBLIN-434 URL: https://issues.apache.org/jira/browse/GOBBLIN-434 Project: Apache Gobblin Issue

[jira] [Created] (GOBBLIN-433) Try to query schema registry for empty workunits

2018-03-21 Thread Tamas Nemeth (JIRA)
Tamas Nemeth created GOBBLIN-433: Summary: Try to query schema registry for empty workunits Key: GOBBLIN-433 URL: https://issues.apache.org/jira/browse/GOBBLIN-433 Project: Apache Gobblin

[jira] [Created] (GOBBLIN-432) Share the DataSource used by the MySQL state stores

2018-03-21 Thread Hung Tran (JIRA)
Hung Tran created GOBBLIN-432: - Summary: Share the DataSource used by the MySQL state stores Key: GOBBLIN-432 URL: https://issues.apache.org/jira/browse/GOBBLIN-432 Project: Apache Gobblin Issue

[VOTE] Apache Gobblin 0.12.0 release RC1

2018-03-21 Thread Abhishek Tiwari
Hi all, I'd like to call a vote to release Apache Gobblin 0.12.0 (Incubating). The previous release candidate RC0 did not pass vote: https://www.mail-archive.com/general@incubator.apache.org/msg62151.html As required, the LICENSE and NOTICE files have been updated (tracked by GOBBLIN-431

[jira] [Updated] (GOBBLIN-431) Update LICENSE and NOTICE for Apache Gobblin

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-431: Description: Update LICENSE and NOTICE for Apache Gobblin. Bring it in accordance to

[jira] [Resolved] (GOBBLIN-431) Update LICENSE and NOTICE for Apache Gobblin

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari resolved GOBBLIN-431. - Resolution: Fixed > Update LICENSE and NOTICE for Apache Gobblin >

[jira] [Commented] (GOBBLIN-431) Update LICENSE and NOTICE for Apache Gobblin

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407916#comment-16407916 ] Abhishek Tiwari commented on GOBBLIN-431: - Relevant commit:

[jira] [Updated] (GOBBLIN-302) Handle stuck Helix workflow

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-302: Fix Version/s: (was: 0.13.0) > Handle stuck Helix workflow >

[jira] [Updated] (GOBBLIN-364) Exclude JobState from WorkUnit created by PartitionedFileSourceBase

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-364: Fix Version/s: (was: 0.13.0) > Exclude JobState from WorkUnit created by

[jira] [Updated] (GOBBLIN-359) Logged Job/Task info from TaskExecutor threads sometimes does not match the task running

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-359: Fix Version/s: (was: 0.13.0) > Logged Job/Task info from TaskExecutor threads

[jira] [Updated] (GOBBLIN-361) Support Nested nullable Record type for JDBCWriter

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-361: Fix Version/s: (was: 0.13.0) > Support Nested nullable Record type for JDBCWriter >

[jira] [Updated] (GOBBLIN-357) Poor logging when zookeeper connection is lost

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-357: Fix Version/s: (was: 0.13.0) > Poor logging when zookeeper connection is lost >

[jira] [Updated] (GOBBLIN-356) hanging when retrieving kafka schema

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-356: Fix Version/s: (was: 0.13.0) > hanging when retrieving kafka schema >

[jira] [Updated] (GOBBLIN-363) Clean up the job-level subdir in the _taskstate directory in Gobblin Cluster after a job is done

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-363: Fix Version/s: (was: 0.13.0) > Clean up the job-level subdir in the _taskstate

[jira] [Updated] (GOBBLIN-351) Add docs for ParquetHdfsDataWriter

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-351: Fix Version/s: (was: 0.13.0) > Add docs for ParquetHdfsDataWriter >

[jira] [Updated] (GOBBLIN-360) Helix not pruning old Zookeeper data

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-360: Fix Version/s: (was: 0.13.0) > Helix not pruning old Zookeeper data >

[jira] [Updated] (GOBBLIN-365) Add lookback days config property for CopyableGlobDatasetFinder

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-365: Fix Version/s: (was: 0.13.0) > Add lookback days config property for

[jira] [Updated] (GOBBLIN-379) Submit an event when DistCp job resource requirements exceed a hard bound.

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-379: Fix Version/s: (was: 0.13.0) > Submit an event when DistCp job resource requirements

[jira] [Updated] (GOBBLIN-378) Ensure task only publish data when the state is successful in the earlier processing

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-378: Fix Version/s: (was: 0.13.0) > Ensure task only publish data when the state is

[jira] [Updated] (GOBBLIN-372) Workaround helix workflow deletion bug that removes workflows with a matching prefix

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-372: Fix Version/s: (was: 0.13.0) > Workaround helix workflow deletion bug that removes

[jira] [Updated] (GOBBLIN-384) Update Python version in gobblin-pr

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-384: Fix Version/s: (was: 0.13.0) > Update Python version in gobblin-pr >

[jira] [Updated] (GOBBLIN-388) Allow classpath to be configured for JVM based task execution in gobblin cluster

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-388: Fix Version/s: (was: 0.13.0) > Allow classpath to be configured for JVM based task

[jira] [Updated] (GOBBLIN-369) Clean up the helix job queue after the job execution is complete

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-369: Fix Version/s: (was: 0.13.0) > Clean up the helix job queue after the job execution is

[jira] [Updated] (GOBBLIN-382) Support storing job.state file in mysql state store for standalone cluster

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-382: Fix Version/s: (was: 0.13.0) > Support storing job.state file in mysql state store for

[jira] [Updated] (GOBBLIN-381) Add ability to filter hidden directories for ConfigBasedDatasets

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-381: Fix Version/s: (was: 0.13.0) > Add ability to filter hidden directories for

[jira] [Updated] (GOBBLIN-377) Add debug logging to print out job configuration in gobblin cluster

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-377: Fix Version/s: (was: 0.13.0) > Add debug logging to print out job configuration in

[jira] [Updated] (GOBBLIN-402) Add more metrics for gobblin cluster and fix the getJobs slowness issue

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-402: Fix Version/s: (was: 0.13.0) > Add more metrics for gobblin cluster and fix the

[jira] [Updated] (GOBBLIN-397) Create a new dataset version selection policy for filtering dataset versions that have "hidden" paths

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-397: Fix Version/s: (was: 0.13.0) > Create a new dataset version selection policy for

[jira] [Updated] (GOBBLIN-398) Upgrade helix to 0.6.9

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-398: Fix Version/s: (was: 0.13.0) > Upgrade helix to 0.6.9 > -- > >

[jira] [Updated] (GOBBLIN-403) Fix the NPE issue due to uninitialized kafkajobmonitor metrics

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-403: Fix Version/s: (was: 0.13.0) > Fix the NPE issue due to uninitialized kafkajobmonitor

[jira] [Updated] (GOBBLIN-396) Date partition based json to avro source

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-396: Fix Version/s: (was: 0.13.0) > Date partition based json to avro source >

[jira] [Updated] (GOBBLIN-399) Refactor HiveSource#shouldCreateWorkunit() to accept table as parameter

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-399: Fix Version/s: (was: 0.13.0) > Refactor HiveSource#shouldCreateWorkunit() to accept

[jira] [Updated] (GOBBLIN-401) Provide a constructor for CombineSelectionPolicy with only the selection config as argument

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-401: Fix Version/s: (was: 0.13.0) > Provide a constructor for CombineSelectionPolicy with

[jira] [Updated] (GOBBLIN-390) Allow child process to be launched with log4j options

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-390: Fix Version/s: (was: 0.13.0) > Allow child process to be launched with log4j options >

[jira] [Updated] (GOBBLIN-391) Use the DataPublisherFactory to allow sharing publishers in SafeDatasetCommit

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-391: Fix Version/s: (was: 0.13.0) > Use the DataPublisherFactory to allow sharing

[jira] [Updated] (GOBBLIN-395) Add lineage for copying config based dataset

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-395: Fix Version/s: (was: 0.13.0) > Add lineage for copying config based dataset >

[jira] [Updated] (GOBBLIN-404) Disable immediate execution of all flows in FlowCatalog on Gobblin Service restart

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-404: Fix Version/s: (was: 0.13.0) > Disable immediate execution of all flows in FlowCatalog

[jira] [Updated] (GOBBLIN-405) Fix race condition with access to immediately invalidated resources

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-405: Fix Version/s: (was: 0.13.0) > Fix race condition with access to immediately

[jira] [Updated] (GOBBLIN-406) [GaaS] Delete job state on spec delete

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-406: Fix Version/s: (was: 0.13.0) > [GaaS] Delete job state on spec delete >

[jira] [Updated] (GOBBLIN-407) Job output is being written to _append directories for full snapshots

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-407: Fix Version/s: (was: 0.13.0) > Job output is being written to _append directories for

[jira] [Updated] (GOBBLIN-413) compaction should use the same time range check

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-413: Fix Version/s: (was: 0.13.0) > compaction should use the same time range check >

[jira] [Updated] (GOBBLIN-414) Add lineage event for convertible hive datasets

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-414: Fix Version/s: (was: 0.13.0) > Add lineage event for convertible hive datasets >

[jira] [Updated] (GOBBLIN-409) Set collation to latin1_bin for the MySql state store backing table

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-409: Fix Version/s: (was: 0.13.0) > Set collation to latin1_bin for the MySql state store

[jira] [Updated] (GOBBLIN-410) Support REPLACE_TABLE_AND_PARTITIONS for Hive copies

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-410: Fix Version/s: (was: 0.13.0) > Support REPLACE_TABLE_AND_PARTITIONS for Hive copies >

[jira] [Updated] (GOBBLIN-419) Add more metrics for cluster job scheduling

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-419: Fix Version/s: (was: 0.13.0) > Add more metrics for cluster job scheduling >

[jira] [Updated] (GOBBLIN-422) FileBasedSource needs fs snapshot update of previously failed workunits with latest snapshot

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-422: Fix Version/s: (was: 0.13.0) > FileBasedSource needs fs snapshot update of previously

[jira] [Updated] (GOBBLIN-424) Gobblin job broker does not get closed if job fails

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-424: Fix Version/s: (was: 0.13.0) > Gobblin job broker does not get closed if job fails >

[jira] [Updated] (GOBBLIN-418) Change Gobblin Service behavior to not call addSpec for pre-existing specs on FlowCatalog start up

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-418: Fix Version/s: (was: 0.13.0) > Change Gobblin Service behavior to not call addSpec for

[jira] [Updated] (GOBBLIN-417) AvroR2JoinConverter passes in the content-type for Rest.li protocol version

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-417: Fix Version/s: (was: 0.13.0) > AvroR2JoinConverter passes in the content-type for

[jira] [Updated] (GOBBLIN-415) Check for the value of configuration key flow.runImmediately in Job config.

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-415: Fix Version/s: (was: 0.13.0) > Check for the value of configuration key

[jira] [Updated] (GOBBLIN-421) Add parameterized type for Pusher message type

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-421: Fix Version/s: (was: 0.13.0) > Add parameterized type for Pusher message type >

[jira] [Updated] (GOBBLIN-416) Allow user to configure java options to launch child process for cluster task isolation

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-416: Fix Version/s: (was: 0.13.0) > Allow user to configure java options to launch child

[jira] [Updated] (GOBBLIN-429) Pass jvm options to child process for task isolation

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-429: Fix Version/s: (was: 0.13.0) > Pass jvm options to child process for task isolation >

[jira] [Updated] (GOBBLIN-431) Update LICENSE and NOTICE for Apache Gobblin

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-431: Fix Version/s: 0.12.0 > Update LICENSE and NOTICE for Apache Gobblin >

[jira] [Created] (GOBBLIN-431) Update LICENSE and NOTICE for Apache Gobblin

2018-03-21 Thread Abhishek Tiwari (JIRA)
Abhishek Tiwari created GOBBLIN-431: --- Summary: Update LICENSE and NOTICE for Apache Gobblin Key: GOBBLIN-431 URL: https://issues.apache.org/jira/browse/GOBBLIN-431 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-351) Add docs for ParquetHdfsDataWriter

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-351: Fix Version/s: 0.12.0 > Add docs for ParquetHdfsDataWriter >

[jira] [Updated] (GOBBLIN-359) Logged Job/Task info from TaskExecutor threads sometimes does not match the task running

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-359: Fix Version/s: 0.12.0 > Logged Job/Task info from TaskExecutor threads sometimes does not

[jira] [Updated] (GOBBLIN-357) Poor logging when zookeeper connection is lost

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-357: Fix Version/s: 0.12.0 > Poor logging when zookeeper connection is lost >

[jira] [Updated] (GOBBLIN-207) Gobblin AWS requires job package to be publicly accessible

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-207: Fix Version/s: 0.12.0 > Gobblin AWS requires job package to be publicly accessible >

[jira] [Updated] (GOBBLIN-358) Add logs for GobblinMetrics

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-358: Fix Version/s: 0.12.0 > Add logs for GobblinMetrics > --- > >

[jira] [Updated] (GOBBLIN-356) hanging when retrieving kafka schema

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-356: Fix Version/s: 0.12.0 > hanging when retrieving kafka schema >

[jira] [Updated] (GOBBLIN-362) Improve DDL on staging table creation for MySQL to also have properties from destination table

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-362: Fix Version/s: 0.12.0 > Improve DDL on staging table creation for MySQL to also have

[jira] [Updated] (GOBBLIN-302) Handle stuck Helix workflow

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-302: Fix Version/s: 0.12.0 > Handle stuck Helix workflow > --- > >

[jira] [Updated] (GOBBLIN-360) Helix not pruning old Zookeeper data

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-360: Fix Version/s: 0.12.0 > Helix not pruning old Zookeeper data >

[jira] [Updated] (GOBBLIN-361) Support Nested nullable Record type for JDBCWriter

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-361: Fix Version/s: 0.12.0 > Support Nested nullable Record type for JDBCWriter >

[jira] [Updated] (GOBBLIN-379) Submit an event when DistCp job resource requirements exceed a hard bound.

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-379: Fix Version/s: 0.12.0 > Submit an event when DistCp job resource requirements exceed a

[jira] [Updated] (GOBBLIN-369) Clean up the helix job queue after the job execution is complete

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-369: Fix Version/s: 0.12.0 > Clean up the helix job queue after the job execution is complete >

[jira] [Updated] (GOBBLIN-378) Ensure task only publish data when the state is successful in the earlier processing

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-378: Fix Version/s: 0.12.0 > Ensure task only publish data when the state is successful in the

[jira] [Updated] (GOBBLIN-365) Add lookback days config property for CopyableGlobDatasetFinder

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-365: Fix Version/s: 0.12.0 > Add lookback days config property for CopyableGlobDatasetFinder >

[jira] [Updated] (GOBBLIN-371) In gobblin_pr, Jira resolution fails if python jira package is not installed

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-371: Fix Version/s: 0.12.0 > In gobblin_pr, Jira resolution fails if python jira package is not

[jira] [Updated] (GOBBLIN-372) Workaround helix workflow deletion bug that removes workflows with a matching prefix

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-372: Fix Version/s: 0.12.0 > Workaround helix workflow deletion bug that removes workflows with

[jira] [Updated] (GOBBLIN-381) Add ability to filter hidden directories for ConfigBasedDatasets

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-381: Fix Version/s: 0.12.0 > Add ability to filter hidden directories for ConfigBasedDatasets >

[jira] [Updated] (GOBBLIN-363) Clean up the job-level subdir in the _taskstate directory in Gobblin Cluster after a job is done

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-363: Fix Version/s: 0.12.0 > Clean up the job-level subdir in the _taskstate directory in

[jira] [Updated] (GOBBLIN-364) Exclude JobState from WorkUnit created by PartitionedFileSourceBase

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-364: Fix Version/s: 0.12.0 > Exclude JobState from WorkUnit created by

[jira] [Updated] (GOBBLIN-397) Create a new dataset version selection policy for filtering dataset versions that have "hidden" paths

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-397: Fix Version/s: 0.12.0 > Create a new dataset version selection policy for filtering

[jira] [Updated] (GOBBLIN-382) Support storing job.state file in mysql state store for standalone cluster

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-382: Fix Version/s: 0.12.0 > Support storing job.state file in mysql state store for standalone

[jira] [Updated] (GOBBLIN-388) Allow classpath to be configured for JVM based task execution in gobblin cluster

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-388: Fix Version/s: 0.12.0 > Allow classpath to be configured for JVM based task execution in

[jira] [Updated] (GOBBLIN-392) Load all dataset states when getLatestDatasetStatesByUrns() is called

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-392: Fix Version/s: 0.12.0 > Load all dataset states when getLatestDatasetStatesByUrns() is

[jira] [Updated] (GOBBLIN-395) Add lineage for copying config based dataset

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-395: Fix Version/s: 0.12.0 > Add lineage for copying config based dataset >

[jira] [Updated] (GOBBLIN-399) Refactor HiveSource#shouldCreateWorkunit() to accept table as parameter

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-399: Fix Version/s: 0.12.0 > Refactor HiveSource#shouldCreateWorkunit() to accept table as

[jira] [Updated] (GOBBLIN-384) Update Python version in gobblin-pr

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-384: Fix Version/s: 0.12.0 > Update Python version in gobblin-pr >

[jira] [Updated] (GOBBLIN-390) Allow child process to be launched with log4j options

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-390: Fix Version/s: 0.12.0 > Allow child process to be launched with log4j options >

[jira] [Updated] (GOBBLIN-409) Set collation to latin1_bin for the MySql state store backing table

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-409: Fix Version/s: 0.12.0 > Set collation to latin1_bin for the MySql state store backing

[jira] [Updated] (GOBBLIN-414) Add lineage event for convertible hive datasets

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-414: Fix Version/s: 0.12.0 > Add lineage event for convertible hive datasets >

[jira] [Updated] (GOBBLIN-411) Fix bug in FIFO based pull file loader

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-411: Fix Version/s: 0.12.0 > Fix bug in FIFO based pull file loader >

[jira] [Updated] (GOBBLIN-418) Change Gobblin Service behavior to not call addSpec for pre-existing specs on FlowCatalog start up

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-418: Fix Version/s: 0.12.0 > Change Gobblin Service behavior to not call addSpec for

[jira] [Updated] (GOBBLIN-413) compaction should use the same time range check

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-413: Fix Version/s: 0.12.0 > compaction should use the same time range check >

[jira] [Updated] (GOBBLIN-419) Add more metrics for cluster job scheduling

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-419: Fix Version/s: 0.12.0 > Add more metrics for cluster job scheduling >

[jira] [Updated] (GOBBLIN-428) Fix delete spec in cluster

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-428: Fix Version/s: 0.12.0 > Fix delete spec in cluster > -- > >

[jira] [Reopened] (GOBBLIN-381) Add ability to filter hidden directories for ConfigBasedDatasets

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari reopened GOBBLIN-381: - > Add ability to filter hidden directories for ConfigBasedDatasets >

[jira] [Resolved] (GOBBLIN-365) Add lookback days config property for CopyableGlobDatasetFinder

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari resolved GOBBLIN-365. - Resolution: Fixed > Add lookback days config property for CopyableGlobDatasetFinder >

[jira] [Updated] (GOBBLIN-429) Pass jvm options to child process for task isolation

2018-03-21 Thread Abhishek Tiwari (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Tiwari updated GOBBLIN-429: Fix Version/s: 0.12.0 > Pass jvm options to child process for task isolation >