[jira] [Resolved] (GOBBLIN-912) Enable TTL caching on Hive Metastore client connection

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-912. -- Resolution: Fixed > Enable TTL caching on Hive Metastore client connection >

[jira] [Resolved] (GOBBLIN-902) Enable gobblin yarn app luncher class configurable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-902. -- Resolution: Fixed > Enable gobblin yarn app luncher class configurable >

[jira] [Resolved] (GOBBLIN-1136) Make LogCopier be able to refresh FileSystem for long running job use cases

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1136. --- Resolution: Fixed > Make LogCopier be able to refresh FileSystem for long running job use cases >

[jira] [Resolved] (GOBBLIN-1147) Use one dfsClient in FsDataWriter to to rename and exists check to avoid inconsistency

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1147. --- Resolution: Fixed > Use one dfsClient in FsDataWriter to to rename and exists check to avoid >

[jira] [Resolved] (GOBBLIN-1133) Add CompactionSuiteBaseWithConfigurableCompleteAction to make complete action configurable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1133. --- Resolution: Fixed > Add CompactionSuiteBaseWithConfigurableCompleteAction to make complete action

[jira] [Resolved] (GOBBLIN-1080) Add configuration to preserve schema creation time in converter

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1080. --- Resolution: Fixed > Add configuration to preserve schema creation time in converter >

[jira] [Resolved] (GOBBLIN-1121) Fix Issue that YarnService use the old token to acquire new container

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1121. --- Resolution: Fixed > Fix Issue that YarnService use the old token to acquire new container >

[jira] [Resolved] (GOBBLIN-1143) Add a generic wrapper producer client to communicate with Kafka

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1143. --- Resolution: Fixed > Add a generic wrapper producer client to communicate with Kafka >

[jira] [Resolved] (GOBBLIN-917) gaas jobs not able to report status should be killed after some time

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-917. -- Resolution: Fixed > gaas jobs not able to report status should be killed after some

[jira] [Resolved] (GOBBLIN-919) fix bad import

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-919. -- Resolution: Fixed > fix bad import > -- > > Key:

[jira] [Resolved] (GOBBLIN-914) accept more tracking event in gobblin service job status monitor

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-914. -- Resolution: Fixed > accept more tracking event in gobblin service job status monitor

[jira] [Resolved] (GOBBLIN-922) fix start sla time unit conversion issue

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-922. -- Resolution: Fixed > fix start sla time unit conversion issue >

[jira] [Resolved] (GOBBLIN-1166) Remove deploy stage from travis yaml file

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1166. --- Resolution: Resolved > Remove deploy stage from travis yaml file >

[jira] [Resolved] (GOBBLIN-1167) Undo gobblin ELR changes

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1167. --- Resolution: Resolved > Undo gobblin ELR changes > > >

[jira] [Resolved] (GOBBLIN-916) Make ContainerLaunchContext instantiation in YarnService more efficient

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-916. -- Resolution: Fixed > Make ContainerLaunchContext instantiation in YarnService more efficient >

[jira] [Resolved] (GOBBLIN-921) Make pull/push mode when registering partition to be configurable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-921. -- Resolution: Fixed > Make pull/push mode when registering partition to be configurable >

[jira] [Resolved] (GOBBLIN-924) Get rid of orc.schema.literal in ORC-ingestion and registration

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-924. -- Resolution: Fixed > Get rid of orc.schema.literal in ORC-ingestion and registration >

[jira] [Resolved] (GOBBLIN-1165) Add config to enable user to set additional yarn classpathes

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1165. --- Resolution: Fixed > Add config to enable user to set additional yarn classpathes >

[jira] [Resolved] (GOBBLIN-1183) Enable additional yarn class path set for app master

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1183. --- Resolution: Fixed > Enable additional yarn class path set for app master >

[jira] [Resolved] (GOBBLIN-1158) Use input dir to document old files instead of file pathes to reduce memory cost in Compaction configurator

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1158. --- Resolution: Fixed > Use input dir to document old files instead of file pathes to reduce memory >

[jira] [Resolved] (GOBBLIN-1085) fix compaction source initialization

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1085. --- Resolution: Fixed > fix compaction source initialization >

[jira] [Resolved] (GOBBLIN-1090) distinguish compiled_skip from compiled_failed metrics

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1090. --- Resolution: Fixed > distinguish compiled_skip from compiled_failed metrics >

[jira] [Resolved] (GOBBLIN-1086) update gaas job start timer using events coming from job execution

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1086. --- Resolution: Fixed > update gaas job start timer using events coming from job

[jira] [Resolved] (GOBBLIN-1027) monitor number of jobs submitted by users

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1027. --- Resolution: Fixed > monitor number of jobs submitted by users >

[jira] [Resolved] (GOBBLIN-1035) make hive dataset descriptor accepts regexed db and tables

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1035. --- Resolution: Fixed > make hive dataset descriptor accepts regexed db and tables >

[jira] [Resolved] (GOBBLIN-1063) add some more info to logs

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1063. --- Resolution: Fixed > add some more info to logs > -- > >

[jira] [Resolved] (GOBBLIN-948) add hive data node and hive descriptor for gobblin service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-948. -- Resolution: Fixed > add hive data node and hive descriptor for gobblin service >

[jira] [Resolved] (GOBBLIN-958) make hive flow edge accept multiple tables

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-958. -- Resolution: Fixed > make hive flow edge accept multiple tables >

[jira] [Resolved] (GOBBLIN-1082) compile a flow before storing it in spec catalog

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1082. --- Resolution: Fixed > compile a flow before storing it in spec catalog >

[jira] [Resolved] (GOBBLIN-1041) send metrics for workunit creation time

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1041. --- Resolution: Fixed > send metrics for workunit creation time >

[jira] [Resolved] (GOBBLIN-1040) Fix High level consumer

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1040. --- Resolution: Resolved > Fix High level consumer > > >

[jira] [Resolved] (GOBBLIN-1159) Publish gobblin artifacts to linkedin Jfrog Bintray

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1159. --- Resolution: Resolved > Publish gobblin artifacts to linkedin Jfrog Bintray >

[jira] [Resolved] (GOBBLIN-1094) Gobblin documentation for High level consumer

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1094. --- Resolution: Resolved > Gobblin documentation for High level consumer >

[jira] [Resolved] (GOBBLIN-1102) Update High Level consumer doc with link to confluence WIKI Gobblin improvement proposals

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1102. --- Resolution: Resolved > Update High Level consumer doc with link to confluence WIKI Gobblin

[jira] [Resolved] (GOBBLIN-1037) Disable UnixTimestampRecursiveCopyableDatasetTest

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1037. --- Resolution: Resolved > Disable UnixTimestampRecursiveCopyableDatasetTest >

[jira] [Resolved] (GOBBLIN-863) Handle race condition between concurrent Gobblin tasks performing Hive registration

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-863. -- Resolution: Fixed > Handle race condition between concurrent Gobblin tasks performing Hive >

[jira] [Resolved] (GOBBLIN-861) Skip getPartition() call to Hive Metastore when a partition already exists

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-861. -- Resolution: Fixed > Skip getPartition() call to Hive Metastore when a partition already exists >

[jira] [Resolved] (GOBBLIN-859) Let writer pass latest schema to workUnitState when schema change

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-859. -- Resolution: Fixed > Let writer pass latest schema to workUnitState when schema change >

[jira] [Resolved] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-899. -- Resolution: Fixed > Add a key in dataset config to disable schema check for a specific dataset >

[jira] [Resolved] (GOBBLIN-877) Add column metadata for partition for inline hive registration

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-877. -- Resolution: Fixed > Add column metadata for partition for inline hive registration >

[jira] [Resolved] (GOBBLIN-1064) Make KafkaAvroSchemaRegistry extendable

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1064. --- Resolution: Fixed > Make KafkaAvroSchemaRegistry extendable >

[jira] [Resolved] (GOBBLIN-1077) Fix bug in HiveDataset.resolveConfig

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1077. --- Resolution: Fixed > Fix bug in HiveDataset.resolveConfig > >

[jira] [Resolved] (GOBBLIN-1069) Add NPE check in handleContainerCompletion method

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1069. --- Resolution: Fixed > Add NPE check in handleContainerCompletion method >

[jira] [Resolved] (GOBBLIN-1199) convert seconds to ms because helix api take time in ms

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1199. --- Resolution: Fixed > convert seconds to ms because helix api take time in ms >

[jira] [Resolved] (GOBBLIN-1152) HelixException in enabling helix instance

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1152. --- Resolution: Fixed > HelixException in enabling helix instance >

[jira] [Resolved] (GOBBLIN-1145) add table/partition path in serde props

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1145. --- Resolution: Fixed > add table/partition path in serde props >

[jira] [Resolved] (GOBBLIN-1150) change spec store table to use json type

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1150. --- Resolution: Fixed > change spec store table to use json type >

[jira] [Resolved] (GOBBLIN-1198) use DatasetCleanerTask to clean job status

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1198. --- Resolution: Fixed > use DatasetCleanerTask to clean job status >

[jira] [Resolved] (GOBBLIN-1168) create InstrumentedSpecStore

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1168. --- Resolution: Fixed > create InstrumentedSpecStore > >

[jira] [Resolved] (GOBBLIN-1157) handle non string type config values

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1157. --- Resolution: Fixed > handle non string type config values >

[jira] [Resolved] (GOBBLIN-1160) do not delete spec on compilation failures at startup

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1160. --- Resolution: Fixed > do not delete spec on compilation failures at startup >

[jira] [Resolved] (GOBBLIN-1144) move spec store delete to gobblinservice job scheduler

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1144. --- Resolution: Fixed > move spec store delete to gobblinservice job scheduler >

[jira] [Resolved] (GOBBLIN-886) add callbak to kafka

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-886. -- Resolution: Fixed > add callbak to kafka > > >

[jira] [Resolved] (GOBBLIN-864) add job error message in job state

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-864. -- Resolution: Fixed > add job error message in job state >

[jira] [Resolved] (GOBBLIN-892) fix a bug where we are setting/getting props in dataset state. it is not a supported operation.

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-892. -- Resolution: Fixed > fix a bug where we are setting/getting props in dataset state. it

[jira] [Work started] (GOBBLIN-892) fix a bug where we are setting/getting props in dataset state. it is not a supported operation.

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on GOBBLIN-892 started by Arjun Singh Bora. > fix a bug where we are setting/getting props in dataset state. it is

[jira] [Resolved] (GOBBLIN-1169) Bintray publishing of gobblin modules

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1169. --- Resolution: Resolved > Bintray publishing of gobblin modules >

[jira] [Closed] (GOBBLIN-1163) Make travis alive again

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra closed GOBBLIN-1163. - Resolution: Done > Make travis alive again > > >

[jira] [Resolved] (GOBBLIN-954) Added logic to support swapping of different implementations of HiveRegistrationPublisher

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-954. -- Resolution: Resolved > Added logic to support swapping of different implementations of >

[jira] [Resolved] (GOBBLIN-910) Add Unix timestamp copyable recursive dataset finder with snapshot selection policy

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-910. -- Resolution: Resolved > Add Unix timestamp copyable recursive dataset finder with snapshot

[jira] [Closed] (GOBBLIN-1164) Remove deploy phase from travis

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra closed GOBBLIN-1164. - Resolution: Invalid > Remove deploy phase from travis > --- > >

[jira] [Resolved] (GOBBLIN-941) Enhance DDL to add column and column.types with case-preserving schema

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-941. -- Resolution: Fixed > Enhance DDL to add column and column.types with case-preserving schema >

[jira] [Resolved] (GOBBLIN-986) Persist the existing property of table when doing hive registration

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-986. -- Resolution: Fixed > Persist the existing property of table when doing hive registration >

[jira] [Resolved] (GOBBLIN-981) Handle backward compatibility issue in HiveSource

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-981. -- Resolution: Fixed > Handle backward compatibility issue in HiveSource >

[jira] [Resolved] (GOBBLIN-975) Add flag to enable/disable avro type check in AvroToOrc

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-975. -- Resolution: Fixed > Add flag to enable/disable avro type check in AvroToOrc >

[jira] [Resolved] (GOBBLIN-1196) add support to search flow configs using multiple configs and other params

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1196. --- Resolution: Fixed > add support to search flow configs using multiple configs and

[jira] [Resolved] (GOBBLIN-1219) do not schedule a flow spec from a slave instance of GobblinServiceJobScheduler

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1219. --- Resolution: Fixed > do not schedule a flow spec from a slave instance of >

[jira] [Resolved] (GOBBLIN-841) make some field public to de duplicate strings

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-841. -- Resolution: Fixed > make some field public to de duplicate strings >

[jira] [Resolved] (GOBBLIN-840) avoid creating a flow execution id by both master and slave gaas

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-840. -- Resolution: Fixed > avoid creating a flow execution id by both master and slave gaas

[jira] [Resolved] (GOBBLIN-808) implement azkaban job cancellation in Gobblin Service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-808. -- Resolution: Fixed > implement azkaban job cancellation in Gobblin Service >

[jira] [Resolved] (GOBBLIN-856) make job status monitor top level service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-856. -- Resolution: Fixed > make job status monitor top level service >

[jira] [Resolved] (GOBBLIN-855) persist the dag after addSpec to spec producer

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-855. -- Resolution: Fixed > persist the dag after addSpec to spec producer >

[jira] [Resolved] (GOBBLIN-847) add a flow level sla in gaas flows

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-847. -- Resolution: Fixed > add a flow level sla in gaas flows >

[jira] [Resolved] (GOBBLIN-852) Reorganize the code for hive registration to isolate function

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-852. -- Resolution: Fixed > Reorganize the code for hive registration to isolate function >

[jira] [Resolved] (GOBBLIN-806) Enable metrics reporter during dataset discovery for retention job

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-806. -- Resolution: Fixed > Enable metrics reporter during dataset discovery for retention job >

[jira] [Resolved] (GOBBLIN-1023) Fix the issue of lossing data when trying to commit

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1023. --- Resolution: Won't Fix > Fix the issue of lossing data when trying to commit >

[jira] [Resolved] (GOBBLIN-1105) make MysqlJobStatusStateStore implements DatasetStateStore

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1105. --- Resolution: Fixed > make MysqlJobStatusStateStore implements DatasetStateStore >

[jira] [Resolved] (GOBBLIN-1092) do not store explain flowConfig requests to spec catalog

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1092. --- Resolution: Fixed > do not store explain flowConfig requests to spec catalog >

[jira] [Resolved] (GOBBLIN-1106) do not remove requester list from flow config

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1106. --- Resolution: Fixed > do not remove requester list from flow config >

[jira] [Resolved] (GOBBLIN-1141) provide an option for common configs in cluster

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1141. --- Resolution: Fixed > provide an option for common configs in cluster >

[jira] [Resolved] (GOBBLIN-1109) partial rollback of PR#2836

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1109. --- Resolution: Fixed > partial rollback of PR#2836 > --- > >

[jira] [Resolved] (GOBBLIN-1110) handle runtime exceptions in executeCancellation implementation of GobblinHelixJobLauncher

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1110. --- Resolution: Fixed > handle runtime exceptions in executeCancellation

[jira] [Resolved] (GOBBLIN-1184) Publish gobblin cluster test to artifactory

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1184. --- Resolution: Resolved > Publish gobblin cluster test to artifactory >

[jira] [Resolved] (GOBBLIN-1171) Fix travis build timeouts

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1171. --- Resolution: Resolved > Fix travis build timeouts > - > >

[jira] [Resolved] (GOBBLIN-1215) Add Retry logic to travis builds

2020-07-24 Thread Vikram Bohra (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Bohra resolved GOBBLIN-1215. --- Resolution: Resolved > Add Retry logic to travis builds >

[jira] [Resolved] (GOBBLIN-1210) Force AM to read from token file to update token when start up

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1210. --- Resolution: Fixed > Force AM to read from token file to update token when start up >

[jira] [Resolved] (GOBBLIN-1185) Enable dataset cleaner to emit kafka events

2020-07-24 Thread Zihan Li (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zihan Li resolved GOBBLIN-1185. --- Resolution: Fixed > Enable dataset cleaner to emit kafka events >

[jira] [Resolved] (GOBBLIN-1220) reduce helix job initialization timeout from long.maxval to 5 minutes

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1220. --- Resolution: Fixed > reduce helix job initialization timeout from long.maxval to 5

[jira] [Resolved] (GOBBLIN-1217) add a few map/reduce task related configs in building metric reporter

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-1217. --- Resolution: Fixed > add a few map/reduce task related configs in building metric

[jira] [Resolved] (GOBBLIN-730) include job start and end time in GaaS job status

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-730. -- Resolution: Fixed > include job start and end time in GaaS job status >

[jira] [Resolved] (GOBBLIN-792) submit a GobblinTrackingEvent when jobs are compiled but not yet orchestrated

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-792. -- Resolution: Fixed > submit a GobblinTrackingEvent when jobs are compiled but not yet

[jira] [Resolved] (GOBBLIN-771) emit a few metrics for gobblin service

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-771. -- Resolution: Fixed > emit a few metrics for gobblin service >

[jira] [Resolved] (GOBBLIN-779) make job status retriever configurable

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-779. -- Resolution: Fixed > make job status retriever configurable >

[jira] [Resolved] (GOBBLIN-837) refactor FlowConfigV2Client and FlowStatusClient to allow child classes modify respective RequestBuilders

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-837. -- Resolution: Fixed > refactor FlowConfigV2Client and FlowStatusClient to allow child

[jira] [Resolved] (GOBBLIN-802) register Gauge in innerMetricsContext

2020-07-24 Thread Arjun Singh Bora (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Singh Bora resolved GOBBLIN-802. -- Resolution: Fixed > register Gauge in innerMetricsContext >

[jira] [Resolved] (GOBBLIN-953) Add scoped config for app launcher created by gobblin service

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-953. -- Resolution: Fixed > Add scoped config for app launcher created by gobblin service >

[jira] [Resolved] (GOBBLIN-782) Add config to inject properties to jobs orchestrated by gobblin service

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-782. -- Resolution: Fixed > Add config to inject properties to jobs orchestrated by gobblin service >

[jira] [Resolved] (GOBBLIN-277) Add a lock to make multihop thread safe

2020-07-24 Thread Kuai Yu (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuai Yu resolved GOBBLIN-277. - Resolution: Fixed > Add a lock to make multihop thread safe > --- >

[jira] [Resolved] (GOBBLIN-1073) Add proxy user and requester quota to GaaS

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1073. --- Resolution: Fixed > Add proxy user and requester quota to GaaS >

[jira] [Resolved] (GOBBLIN-1115) Add flow level data movement authorization in gaas

2020-07-24 Thread Jack Moseley (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jack Moseley resolved GOBBLIN-1115. --- Resolution: Fixed > Add flow level data movement authorization in gaas >

  1   2   >