[jira] [Resolved] (GOBBLIN-277) Add a lock to make multihop thread safe

2020-07-24 Thread Kuai Yu (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuai Yu resolved GOBBLIN-277. - Resolution: Fixed > Add a lock to make multihop thread safe > --- >

[jira] [Resolved] (GOBBLIN-373) Expose task executor auto scale metrics to external sensor

2020-07-24 Thread Kuai Yu (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuai Yu resolved GOBBLIN-373. - Resolution: Fixed > Expose task executor auto scale metrics to external sensor >

[jira] [Resolved] (GOBBLIN-953) Add scoped config for app launcher created by gobblin service

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-953. -- Resolution: Fixed > Add scoped config for app launcher created by gobblin service >

[jira] [Resolved] (GOBBLIN-782) Add config to inject properties to jobs orchestrated by gobblin service

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-782. -- Resolution: Fixed > Add config to inject properties to jobs orchestrated by gobblin service >

[jira] [Resolved] (GOBBLIN-969) Bump orc version for bug fix

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-969. -- Resolution: Fixed > Bump orc version for bug fix > > >

[jira] [Resolved] (GOBBLIN-956) Change DagManagerThread to continue running until queue of dags is drained

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-956. -- Resolution: Fixed > Change DagManagerThread to continue running until queue of dags is

[jira] [Resolved] (GOBBLIN-990) Don't allow creation of flow config that already exists

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-990. -- Resolution: Fixed > Don't allow creation of flow config that already exists >

[jira] [Resolved] (GOBBLIN-1050) Verify requester when updating/deleting FlowConfig

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1050. --- Resolution: Fixed > Verify requester when updating/deleting FlowConfig >

[jira] [Resolved] (GOBBLIN-997) Add serialVersionUID to FlowSpec for backwards compatibility

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-997. -- Resolution: Fixed > Add serialVersionUID to FlowSpec for backwards compatibility >

[jira] [Resolved] (GOBBLIN-974) Avoid updating job/flow status if messages arrive out of order

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-974. -- Resolution: Fixed > Avoid updating job/flow status if messages arrive out of order >

[jira] [Resolved] (GOBBLIN-1038) Set default dataset descriptor configs based on the DataNode

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1038. --- Resolution: Fixed > Set default dataset descriptor configs based on the DataNode >

[jira] [Resolved] (GOBBLIN-1062) Add log when loading dags from state store

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1062. --- Resolution: Fixed > Add log when loading dags from state store >

[jira] [Resolved] (GOBBLIN-1019) Change jcenter url to https

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1019. --- Resolution: Fixed > Change jcenter url to https > --- > >

[jira] [Resolved] (GOBBLIN-980) AzkabanClient always throwing exception when cancelling flow

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-980. -- Resolution: Fixed > AzkabanClient always throwing exception when cancelling flow >

[jira] [Resolved] (GOBBLIN-1017) Deprecate FlowStatus in favor of FlowExecution and add endpoint to kill flows

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1017. --- Resolution: Fixed > Deprecate FlowStatus in favor of FlowExecution and add endpoint to

[jira] [Resolved] (GOBBLIN-1130) Add API for adding proxy user to azkaban project

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1130. --- Resolution: Fixed > Add API for adding proxy user to azkaban project >

[jira] [Resolved] (GOBBLIN-1088) Don't lowercase partition pattern config

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1088. --- Resolution: Fixed > Don't lowercase partition pattern config >

[jira] [Resolved] (GOBBLIN-1075) Add option to return latest failed flows

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1075. --- Resolution: Fixed > Add option to return latest failed flows >

[jira] [Resolved] (GOBBLIN-1084) Refresh flowgraph when templates are modified

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1084. --- Resolution: Fixed > Refresh flowgraph when templates are modified >

[jira] [Resolved] (GOBBLIN-1113) Carry forward requester list property when updating flowconfig

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1113. --- Resolution: Fixed > Carry forward requester list property when updating flowconfig >

[jira] [Resolved] (GOBBLIN-1074) Sort job status array when returning flow status

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1074. --- Resolution: Fixed > Sort job status array when returning flow status >

[jira] [Resolved] (GOBBLIN-1073) Add proxy user and requester quota to GaaS

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1073. --- Resolution: Fixed > Add proxy user and requester quota to GaaS >

[jira] [Resolved] (GOBBLIN-1149) Abstract out method for constructing descriptor from config

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1149. --- Resolution: Fixed > Abstract out method for constructing descriptor from config >

[jira] [Resolved] (GOBBLIN-1137) Add API for getting list of proxy users from an azkaban project

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1137. --- Resolution: Fixed > Add API for getting list of proxy users from an azkaban project >

[jira] [Resolved] (GOBBLIN-1115) Add flow level data movement authorization in gaas

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1115. --- Resolution: Fixed > Add flow level data movement authorization in gaas >

[jira] [Resolved] (GOBBLIN-1154) Improve gaas error messages

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1154. --- Resolution: Fixed > Improve gaas error messages > --- > >

[jira] [Resolved] (GOBBLIN-1090) distinguish compiled_skip from compiled_failed metrics

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1090. --- Resolution: Fixed > distinguish compiled_skip from compiled_failed metrics >

[jira] [Resolved] (GOBBLIN-1086) update gaas job start timer using events coming from job execution

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1086. --- Resolution: Fixed > update gaas job start timer using events coming from job

[jira] [Resolved] (GOBBLIN-1085) fix compaction source initialization

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1085. --- Resolution: Fixed > fix compaction source initialization >

[jira] [Resolved] (GOBBLIN-1041) send metrics for workunit creation time

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1041. --- Resolution: Fixed > send metrics for workunit creation time >

[jira] [Resolved] (GOBBLIN-958) make hive flow edge accept multiple tables

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-958. -- Resolution: Fixed > make hive flow edge accept multiple tables >

[jira] [Resolved] (GOBBLIN-1082) compile a flow before storing it in spec catalog

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1082. --- Resolution: Fixed > compile a flow before storing it in spec catalog >

[jira] [Resolved] (GOBBLIN-948) add hive data node and hive descriptor for gobblin service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-948. -- Resolution: Fixed > add hive data node and hive descriptor for gobblin service >

[jira] [Resolved] (GOBBLIN-1027) monitor number of jobs submitted by users

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1027. --- Resolution: Fixed > monitor number of jobs submitted by users >

[jira] [Resolved] (GOBBLIN-1035) make hive dataset descriptor accepts regexed db and tables

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1035. --- Resolution: Fixed > make hive dataset descriptor accepts regexed db and tables >

[jira] [Resolved] (GOBBLIN-1063) add some more info to logs

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1063. --- Resolution: Fixed > add some more info to logs > -- > >

[jira] [Resolved] (GOBBLIN-922) fix start sla time unit conversion issue

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-922. -- Resolution: Fixed > fix start sla time unit conversion issue >

[jira] [Resolved] (GOBBLIN-914) accept more tracking event in gobblin service job status monitor

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-914. -- Resolution: Fixed > accept more tracking event in gobblin service job status monitor

[jira] [Resolved] (GOBBLIN-917) gaas jobs not able to report status should be killed after some time

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-917. -- Resolution: Fixed > gaas jobs not able to report status should be killed after some

[jira] [Resolved] (GOBBLIN-919) fix bad import

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-919. -- Resolution: Fixed > fix bad import > -- > > Key:

[jira] [Resolved] (GOBBLIN-892) fix a bug where we are setting/getting props in dataset state. it is not a supported operation.

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-892. -- Resolution: Fixed > fix a bug where we are setting/getting props in dataset state. it

[jira] [Resolved] (GOBBLIN-886) add callbak to kafka

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-886. -- Resolution: Fixed > add callbak to kafka > > >

[jira] [Resolved] (GOBBLIN-864) add job error message in job state

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-864. -- Resolution: Fixed > add job error message in job state >

[jira] [Work started] (GOBBLIN-892) fix a bug where we are setting/getting props in dataset state. it is not a supported operation.

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on GOBBLIN-892 started by Arjun Singh Bora. > fix a bug where we are setting/getting props in dataset state. it is

[jira] [Resolved] (GOBBLIN-855) persist the dag after addSpec to spec producer

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-855. -- Resolution: Fixed > persist the dag after addSpec to spec producer >

[jira] [Resolved] (GOBBLIN-847) add a flow level sla in gaas flows

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-847. -- Resolution: Fixed > add a flow level sla in gaas flows >

[jira] [Resolved] (GOBBLIN-856) make job status monitor top level service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-856. -- Resolution: Fixed > make job status monitor top level service >

[jira] [Resolved] (GOBBLIN-841) make some field public to de duplicate strings

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-841. -- Resolution: Fixed > make some field public to de duplicate strings >

[jira] [Resolved] (GOBBLIN-840) avoid creating a flow execution id by both master and slave gaas

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-840. -- Resolution: Fixed > avoid creating a flow execution id by both master and slave gaas

[jira] [Resolved] (GOBBLIN-808) implement azkaban job cancellation in Gobblin Service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-808. -- Resolution: Fixed > implement azkaban job cancellation in Gobblin Service >

[jira] [Resolved] (GOBBLIN-837) refactor FlowConfigV2Client and FlowStatusClient to allow child classes modify respective RequestBuilders

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-837. -- Resolution: Fixed > refactor FlowConfigV2Client and FlowStatusClient to allow child

[jira] [Resolved] (GOBBLIN-779) make job status retriever configurable

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-779. -- Resolution: Fixed > make job status retriever configurable >

[jira] [Resolved] (GOBBLIN-792) submit a GobblinTrackingEvent when jobs are compiled but not yet orchestrated

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-792. -- Resolution: Fixed > submit a GobblinTrackingEvent when jobs are compiled but not yet

[jira] [Resolved] (GOBBLIN-771) emit a few metrics for gobblin service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-771. -- Resolution: Fixed > emit a few metrics for gobblin service >

[jira] [Resolved] (GOBBLIN-802) register Gauge in innerMetricsContext

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-802. -- Resolution: Fixed > register Gauge in innerMetricsContext >

[jira] [Resolved] (GOBBLIN-730) include job start and end time in GaaS job status

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-730. -- Resolution: Fixed > include job start and end time in GaaS job status >

[jira] [Resolved] (GOBBLIN-1105) make MysqlJobStatusStateStore implements DatasetStateStore

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1105. --- Resolution: Fixed > make MysqlJobStatusStateStore implements DatasetStateStore >

[jira] [Resolved] (GOBBLIN-1092) do not store explain flowConfig requests to spec catalog

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1092. --- Resolution: Fixed > do not store explain flowConfig requests to spec catalog >

[jira] [Resolved] (GOBBLIN-1110) handle runtime exceptions in executeCancellation implementation of GobblinHelixJobLauncher

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1110. --- Resolution: Fixed > handle runtime exceptions in executeCancellation

[jira] [Resolved] (GOBBLIN-1109) partial rollback of PR#2836

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1109. --- Resolution: Fixed > partial rollback of PR#2836 > --- > >

[jira] [Resolved] (GOBBLIN-1106) do not remove requester list from flow config

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1106. --- Resolution: Fixed > do not remove requester list from flow config >

[jira] [Resolved] (GOBBLIN-1141) provide an option for common configs in cluster

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1141. --- Resolution: Fixed > provide an option for common configs in cluster >

[jira] [Resolved] (GOBBLIN-1150) change spec store table to use json type

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1150. --- Resolution: Fixed > change spec store table to use json type >

[jira] [Resolved] (GOBBLIN-1152) HelixException in enabling helix instance

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1152. --- Resolution: Fixed > HelixException in enabling helix instance >

[jira] [Resolved] (GOBBLIN-1145) add table/partition path in serde props

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1145. --- Resolution: Fixed > add table/partition path in serde props >

[jira] [Resolved] (GOBBLIN-1157) handle non string type config values

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1157. --- Resolution: Fixed > handle non string type config values >

[jira] [Resolved] (GOBBLIN-1160) do not delete spec on compilation failures at startup

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1160. --- Resolution: Fixed > do not delete spec on compilation failures at startup >

[jira] [Resolved] (GOBBLIN-1144) move spec store delete to gobblinservice job scheduler

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1144. --- Resolution: Fixed > move spec store delete to gobblinservice job scheduler >

[jira] [Resolved] (GOBBLIN-1198) use DatasetCleanerTask to clean job status

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1198. --- Resolution: Fixed > use DatasetCleanerTask to clean job status >

[jira] [Resolved] (GOBBLIN-1168) create InstrumentedSpecStore

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1168. --- Resolution: Fixed > create InstrumentedSpecStore > >

[jira] [Resolved] (GOBBLIN-1199) convert seconds to ms because helix api take time in ms

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1199. --- Resolution: Fixed > convert seconds to ms because helix api take time in ms >

[jira] [Resolved] (GOBBLIN-1196) add support to search flow configs using multiple configs and other params

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1196. --- Resolution: Fixed > add support to search flow configs using multiple configs and

[jira] [Resolved] (GOBBLIN-1219) do not schedule a flow spec from a slave instance of GobblinServiceJobScheduler

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1219. --- Resolution: Fixed > do not schedule a flow spec from a slave instance of >

[jira] [Resolved] (GOBBLIN-1220) reduce helix job initialization timeout from long.maxval to 5 minutes

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1220. --- Resolution: Fixed > reduce helix job initialization timeout from long.maxval to 5

[jira] [Resolved] (GOBBLIN-1217) add a few map/reduce task related configs in building metric reporter

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1217. --- Resolution: Fixed > add a few map/reduce task related configs in building metric

[jira] [Resolved] (GOBBLIN-1210) Force AM to read from token file to update token when start up

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1210. --- Resolution: Fixed > Force AM to read from token file to update token when start up >

[jira] [Resolved] (GOBBLIN-1185) Enable dataset cleaner to emit kafka events

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1185. --- Resolution: Fixed > Enable dataset cleaner to emit kafka events >

[jira] [Resolved] (GOBBLIN-1165) Add config to enable user to set additional yarn classpathes

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1165. --- Resolution: Fixed > Add config to enable user to set additional yarn classpathes >

[jira] [Resolved] (GOBBLIN-1183) Enable additional yarn class path set for app master

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1183. --- Resolution: Fixed > Enable additional yarn class path set for app master >

[jira] [Resolved] (GOBBLIN-1158) Use input dir to document old files instead of file pathes to reduce memory cost in Compaction configurator

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1158. --- Resolution: Fixed > Use input dir to document old files instead of file pathes to reduce memory >

[jira] [Resolved] (GOBBLIN-1147) Use one dfsClient in FsDataWriter to to rename and exists check to avoid inconsistency

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1147. --- Resolution: Fixed > Use one dfsClient in FsDataWriter to to rename and exists check to avoid >

[jira] [Resolved] (GOBBLIN-1136) Make LogCopier be able to refresh FileSystem for long running job use cases

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1136. --- Resolution: Fixed > Make LogCopier be able to refresh FileSystem for long running job use cases >

[jira] [Resolved] (GOBBLIN-1143) Add a generic wrapper producer client to communicate with Kafka

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1143. --- Resolution: Fixed > Add a generic wrapper producer client to communicate with Kafka >

[jira] [Resolved] (GOBBLIN-1133) Add CompactionSuiteBaseWithConfigurableCompleteAction to make complete action configurable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1133. --- Resolution: Fixed > Add CompactionSuiteBaseWithConfigurableCompleteAction to make complete action

[jira] [Resolved] (GOBBLIN-1121) Fix Issue that YarnService use the old token to acquire new container

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1121. --- Resolution: Fixed > Fix Issue that YarnService use the old token to acquire new container >

[jira] [Resolved] (GOBBLIN-1080) Add configuration to preserve schema creation time in converter

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1080. --- Resolution: Fixed > Add configuration to preserve schema creation time in converter >

[jira] [Resolved] (GOBBLIN-1069) Add NPE check in handleContainerCompletion method

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1069. --- Resolution: Fixed > Add NPE check in handleContainerCompletion method >

[jira] [Resolved] (GOBBLIN-1064) Make KafkaAvroSchemaRegistry extendable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1064. --- Resolution: Fixed > Make KafkaAvroSchemaRegistry extendable >

[jira] [Resolved] (GOBBLIN-1077) Fix bug in HiveDataset.resolveConfig

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1077. --- Resolution: Fixed > Fix bug in HiveDataset.resolveConfig > >

[jira] [Resolved] (GOBBLIN-1023) Fix the issue of lossing data when trying to commit

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1023. --- Resolution: Won't Fix > Fix the issue of lossing data when trying to commit >

[jira] [Resolved] (GOBBLIN-981) Handle backward compatibility issue in HiveSource

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-981. -- Resolution: Fixed > Handle backward compatibility issue in HiveSource >

[jira] [Resolved] (GOBBLIN-986) Persist the existing property of table when doing hive registration

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-986. -- Resolution: Fixed > Persist the existing property of table when doing hive registration >

[jira] [Resolved] (GOBBLIN-975) Add flag to enable/disable avro type check in AvroToOrc

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-975. -- Resolution: Fixed > Add flag to enable/disable avro type check in AvroToOrc >

[jira] [Resolved] (GOBBLIN-941) Enhance DDL to add column and column.types with case-preserving schema

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-941. -- Resolution: Fixed > Enhance DDL to add column and column.types with case-preserving schema >

[jira] [Resolved] (GOBBLIN-924) Get rid of orc.schema.literal in ORC-ingestion and registration

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-924. -- Resolution: Fixed > Get rid of orc.schema.literal in ORC-ingestion and registration >

[jira] [Resolved] (GOBBLIN-916) Make ContainerLaunchContext instantiation in YarnService more efficient

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-916. -- Resolution: Fixed > Make ContainerLaunchContext instantiation in YarnService more efficient >

[jira] [Resolved] (GOBBLIN-921) Make pull/push mode when registering partition to be configurable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-921. -- Resolution: Fixed > Make pull/push mode when registering partition to be configurable >

[jira] [Resolved] (GOBBLIN-912) Enable TTL caching on Hive Metastore client connection

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-912. -- Resolution: Fixed > Enable TTL caching on Hive Metastore client connection >

[jira] [Resolved] (GOBBLIN-902) Enable gobblin yarn app luncher class configurable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-902. -- Resolution: Fixed > Enable gobblin yarn app luncher class configurable >

[jira] [Resolved] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-899. -- Resolution: Fixed > Add a key in dataset config to disable schema check for a specific dataset >

  1   2   >