[jira] [Resolved] (HIVE-25355) EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails
[ https://issues.apache.org/jira/browse/HIVE-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25355. Resolution: Fixed > EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails > - > > Key: HIVE-25355 > URL: https://issues.apache.org/jira/browse/HIVE-25355 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25579) LOAD overwrite appends rather than overwriting
[
https://issues.apache.org/jira/browse/HIVE-25579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aasha Medhi resolved HIVE-25579.
Resolution: Fixed
> LOAD overwrite appends rather than overwriting
> --
>
> Key: HIVE-25579
> URL: https://issues.apache.org/jira/browse/HIVE-25579
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The overwrite query gets converted to append.
> {noformat}
> 7b6-4b43-8452-52c44e8a2f71): LOAD DATA INPATH
> 'hdfs://ayushsaxena-1.ayushsaxena.root.hwx.site:8020/warehouse/tablespace/external/hive/test_ext/00_0'
> OVERWRITE INTO TABLE test_spark
> 2021-09-30 03:30:23,033 INFO org.apache.hadoop.hive.ql.lockmgr.DbTxnManager:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Opened txnid:15
> 2021-09-30 03:30:23,035 INFO
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Starting caching scope for:
> hive_20210930033023_bb1f6dc4-d7b6-4b43-8452-52c44e8a2f71
> 2021-09-30 03:30:23,042 INFO
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Load data triggered a Tez job instead of usual file operation
> 2021-09-30 03:30:23,042 INFO
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Going to reparse 'hdfs://ayushsaxena-1.ayushsaxena.root.hwx.site:8020/warehouse/tablespace/external/hive/test_ext/00_0'
> OVERWRITE INTO TABLE test_spark> as
> test_spark__temp_table_for_load_data__>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25579) LOAD overwrite appends rather than overwriting
[
https://issues.apache.org/jira/browse/HIVE-25579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423759#comment-17423759
]
Aasha Medhi commented on HIVE-25579:
Thank you for the patch [~ayushtkn] and the review [~anuragshekhar]
> LOAD overwrite appends rather than overwriting
> --
>
> Key: HIVE-25579
> URL: https://issues.apache.org/jira/browse/HIVE-25579
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The overwrite query gets converted to append.
> {noformat}
> 7b6-4b43-8452-52c44e8a2f71): LOAD DATA INPATH
> 'hdfs://ayushsaxena-1.ayushsaxena.root.hwx.site:8020/warehouse/tablespace/external/hive/test_ext/00_0'
> OVERWRITE INTO TABLE test_spark
> 2021-09-30 03:30:23,033 INFO org.apache.hadoop.hive.ql.lockmgr.DbTxnManager:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Opened txnid:15
> 2021-09-30 03:30:23,035 INFO
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Starting caching scope for:
> hive_20210930033023_bb1f6dc4-d7b6-4b43-8452-52c44e8a2f71
> 2021-09-30 03:30:23,042 INFO
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Load data triggered a Tez job instead of usual file operation
> 2021-09-30 03:30:23,042 INFO
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer:
> [db2ab9c9-bf54-4304-bc06-e3bef76f2e79 HiveServer2-Handler-Pool: Thread-2600]:
> Going to reparse 'hdfs://ayushsaxena-1.ayushsaxena.root.hwx.site:8020/warehouse/tablespace/external/hive/test_ext/00_0'
> OVERWRITE INTO TABLE test_spark> as
> test_spark__temp_table_for_load_data__>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25538) CommitTxn replay failing during incremental run
[
https://issues.apache.org/jira/browse/HIVE-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aasha Medhi resolved HIVE-25538.
Resolution: Fixed
> CommitTxn replay failing during incremental run
> ---
>
> Key: HIVE-25538
> URL: https://issues.apache.org/jira/browse/HIVE-25538
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> CommitTxn Fails during incremental run, in case the source file is deleted
> post copy & before checksum validation.
> {noformat}
> 2021-09-21T07:53:40,898 ERROR [TThreadPoolServer WorkerProcess-%d]
> thrift.ProcessFunction: Internal error processing commit_txn
> org.apache.thrift.TException:
> /warehouse1/replicated_testreplcommittransactiononsourcedelete_1632235978675.db/testreplcommittransactiononsourcedelete/load_date=2016-03-01/delta_002_002_
> (is not a directory)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.resolvePath(FSDirectory.java:677)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:151)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1927)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:738)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> at
> org.apache.hadoop.hive.metastore.HMSHandler.commit_txn(HMSHandler.java:8652)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_261]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_261]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy55.commit_txn(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$commit_txn.getResult(ThriftHiveMetastore.java:23159)
> ~[hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$commit_txn.getResult(ThriftHiveMetastore.java:23138)
> ~[hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_261]
> at
>
[jira] [Work started] (HIVE-25355) EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails
[ https://issues.apache.org/jira/browse/HIVE-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-25355 started by Aasha Medhi. -- > EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails > - > > Key: HIVE-25355 > URL: https://issues.apache.org/jira/browse/HIVE-25355 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-25355) EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails
[ https://issues.apache.org/jira/browse/HIVE-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423203#comment-17423203 ] Aasha Medhi edited comment on HIVE-25355 at 10/1/21, 9:33 AM: -- Excluding explain from read only transaction list. This needs to be reverted once https://issues.apache.org/jira/browse/HIVE-25011 is committed was (Author: aasha): Excluding explain from read only transaction list > EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails > - > > Key: HIVE-25355 > URL: https://issues.apache.org/jira/browse/HIVE-25355 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25355) EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails
[ https://issues.apache.org/jira/browse/HIVE-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi reassigned HIVE-25355: -- Assignee: Aasha Medhi (was: Pravin Sinha) > EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails > - > > Key: HIVE-25355 > URL: https://issues.apache.org/jira/browse/HIVE-25355 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25355) EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails
[ https://issues.apache.org/jira/browse/HIVE-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423203#comment-17423203 ] Aasha Medhi commented on HIVE-25355: Excluding explain from read only transaction list > EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails > - > > Key: HIVE-25355 > URL: https://issues.apache.org/jira/browse/HIVE-25355 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25534) Error when executing DistCp on file system not supporting XAttrs
[
https://issues.apache.org/jira/browse/HIVE-25534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aasha Medhi resolved HIVE-25534.
Resolution: Fixed
> Error when executing DistCp on file system not supporting XAttrs
>
>
> Key: HIVE-25534
> URL: https://issues.apache.org/jira/browse/HIVE-25534
> Project: Hive
> Issue Type: Bug
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Remove the preserve xattr while calling distcp.
> {code:java}
> 2021-08-23 10:06:18,485 ERROR org.apache.hadoop.tools.DistCp:
> [HiveServer2-Background-Pool: Thread-73]: XAttrs not supported on at least
> one file system:
> org.apache.hadoop.tools.CopyListing$XAttrsNotSupportedException: XAttrs not
> supported for file system: s3a://hmangla1-dev
> at
> org.apache.hadoop.tools.util.DistCpUtils.checkFileSystemXAttrSupport(DistCpUtils.java:513)
> ~[hadoop-distcp-3.1.1.7.1.6.0-297.jar:?]
> at org.apache.hadoop.tools.DistCp.configureOutputFormat(DistCp.java:337)
> ~[hadoop-distcp-3.1.1.7.1.6.0-297.jar:?]
> at org.apache.hadoop.tools.DistCp.createJob(DistCp.java:304)
> ~[hadoop-distcp-3.1.1.7.1.6.0-297.jar:?]
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:214)
> ~[hadoop-distcp-3.1.1.7.1.6.0-297.jar:?]
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:193)
> ~[hadoop-distcp-3.1.1.7.1.6.0-297.jar:?]{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25330) Make FS calls in CopyUtils retryable
[ https://issues.apache.org/jira/browse/HIVE-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25330. Resolution: Fixed > Make FS calls in CopyUtils retryable > > > Key: HIVE-25330 > URL: https://issues.apache.org/jira/browse/HIVE-25330 > Project: Hive > Issue Type: Improvement >Reporter: Pravin Sinha >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25330) Make FS calls in CopyUtils retryable
[ https://issues.apache.org/jira/browse/HIVE-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419267#comment-17419267 ] Aasha Medhi commented on HIVE-25330: Committed to master. Thank you for the patch [~haymant] and the review [~ayushsaxena] > Make FS calls in CopyUtils retryable > > > Key: HIVE-25330 > URL: https://issues.apache.org/jira/browse/HIVE-25330 > Project: Hive > Issue Type: Improvement >Reporter: Pravin Sinha >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24766) Fix TestScheduledReplication
[ https://issues.apache.org/jira/browse/HIVE-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17417465#comment-17417465 ] Aasha Medhi commented on HIVE-24766: Committed to master. Thank you for the patch [~haymant] and review [~^sharma] > Fix TestScheduledReplication > > > Key: HIVE-24766 > URL: https://issues.apache.org/jira/browse/HIVE-24766 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > test seems to be unstable > http://ci.hive.apache.org/job/hive-flaky-check/184/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24766) Fix TestScheduledReplication
[ https://issues.apache.org/jira/browse/HIVE-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24766. Resolution: Fixed > Fix TestScheduledReplication > > > Key: HIVE-24766 > URL: https://issues.apache.org/jira/browse/HIVE-24766 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > test seems to be unstable > http://ci.hive.apache.org/job/hive-flaky-check/184/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22247) HiveHFileOutputFormat throws FileNotFoundException when partition's task output empty
[
https://issues.apache.org/jira/browse/HIVE-22247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17411865#comment-17411865
]
Aasha Medhi commented on HIVE-22247:
Thank you for the patch [~ayushtkn]. Committed to master.
> HiveHFileOutputFormat throws FileNotFoundException when partition's task
> output empty
> -
>
> Key: HIVE-22247
> URL: https://issues.apache.org/jira/browse/HIVE-22247
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.2.0, 3.0.0
>Reporter: xiepengjie
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When partition's task output empty, HiveHFileOutputFormat throws
> FileNotFoundException like this:
> {code:java}
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: 1 finished. closing...
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: FS[1]: records written - 0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,915 INFO [main]
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
> Committer Algorithm version is 1
> 2019-09-24 19:15:55,954 INFO [main]
> org.apache.hadoop.conf.Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 2019-09-24 19:15:56,089 ERROR [main] ExecReducer: Hit error while closing
> operators - failing tree
> 2019-09-24 19:15:56,090 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1923)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:200)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1016)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:278)
> ... 7 more
> Caused by: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:880)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:109)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:938)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:934)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:945)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1592)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1632)
> at
>
[jira] [Resolved] (HIVE-22247) HiveHFileOutputFormat throws FileNotFoundException when partition's task output empty
[
https://issues.apache.org/jira/browse/HIVE-22247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aasha Medhi resolved HIVE-22247.
Resolution: Fixed
> HiveHFileOutputFormat throws FileNotFoundException when partition's task
> output empty
> -
>
> Key: HIVE-22247
> URL: https://issues.apache.org/jira/browse/HIVE-22247
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.2.0, 3.0.0
>Reporter: xiepengjie
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When partition's task output empty, HiveHFileOutputFormat throws
> FileNotFoundException like this:
> {code:java}
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: 1 finished. closing...
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: FS[1]: records written - 0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,915 INFO [main]
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
> Committer Algorithm version is 1
> 2019-09-24 19:15:55,954 INFO [main]
> org.apache.hadoop.conf.Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 2019-09-24 19:15:56,089 ERROR [main] ExecReducer: Hit error while closing
> operators - failing tree
> 2019-09-24 19:15:56,090 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1923)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:200)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1016)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:278)
> ... 7 more
> Caused by: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:880)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:109)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:938)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:934)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:945)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1592)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1632)
> at
>
[jira] [Resolved] (HIVE-25350) Replication fails for external tables on setting owner/groups
[ https://issues.apache.org/jira/browse/HIVE-25350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25350. Resolution: Fixed > Replication fails for external tables on setting owner/groups > - > > Key: HIVE-25350 > URL: https://issues.apache.org/jira/browse/HIVE-25350 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > DirCopyTask tries to preserve user group permissions, irrespective whether > they have been specified to be preserved or not. > Changing user/group requires SuperUser privileges, hence the task fails. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25350) Replication fails for external tables on setting owner/groups
[ https://issues.apache.org/jira/browse/HIVE-25350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395963#comment-17395963 ] Aasha Medhi commented on HIVE-25350: Thank you for the patch [~ayushtkn] and the review [~^sharma] > Replication fails for external tables on setting owner/groups > - > > Key: HIVE-25350 > URL: https://issues.apache.org/jira/browse/HIVE-25350 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > DirCopyTask tries to preserve user group permissions, irrespective whether > they have been specified to be preserved or not. > Changing user/group requires SuperUser privileges, hence the task fails. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25374) Fix TestReplicationOnHDFSEncryptedZones
[ https://issues.apache.org/jira/browse/HIVE-25374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388687#comment-17388687 ] Aasha Medhi commented on HIVE-25374: Thank you for the patch [~^sharma] and the review [~ayushsaxena]. Committed to master > Fix TestReplicationOnHDFSEncryptedZones > --- > > Key: HIVE-25374 > URL: https://issues.apache.org/jira/browse/HIVE-25374 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25374) Fix TestReplicationOnHDFSEncryptedZones
[ https://issues.apache.org/jira/browse/HIVE-25374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25374. Resolution: Fixed > Fix TestReplicationOnHDFSEncryptedZones > --- > > Key: HIVE-25374 > URL: https://issues.apache.org/jira/browse/HIVE-25374 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22626) Fix Replication related tests
[
https://issues.apache.org/jira/browse/HIVE-22626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383742#comment-17383742
]
Aasha Medhi edited comment on HIVE-22626 at 7/20/21, 4:33 AM:
--
+1
Committed to master.
Thank you for the patch [~ayushtkn] and the review [~^sharma]]
was (Author: aasha):
+1
Committed to master.
Thank you for the patch [~ayushtkn] and the review [~^sharma]
> Fix Replication related tests
> -
>
> Key: HIVE-22626
> URL: https://issues.apache.org/jira/browse/HIVE-22626
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Attachments: qalogs.tgz
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> For TestStatsReplicationScenariosACIDNoAutogather:
> this test is running "alone" because but still; it sometimes runs more than
> 40m which results in a timeout
> a jira search reveals that was pretty common:
>
> [https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc]
> from the hive logs:
> * it seems like after a few minutes this test starts there is an exception:
> {code:java}
> 2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient:
> Unable to shutdown metastore client. Will try closing transport directly.
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Socket closed
> at
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
> ~[libfb303-0.9.3.jar:?]
> at
> com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
> ~[libfb303-0.9.3.jar:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_102]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy62.close(Unknown Source) [?:?]
> at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:542)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.metadata.Hive.finalize(Hive.java:514)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.lang.System$2.invokeFinalize(System.java:1270) [?:1.8.0_102]
> at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
> [?:1.8.0_102]
> at java.lang.ref.Finalizer.access$100(Finalizer.java:34) [?:1.8.0_102]
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
> [?:1.8.0_102]
> Caused by: java.net.SocketException: Socket closed
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
> ~[?:1.8.0_102]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[?:1.8.0_102]
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> ~[?:1.8.0_102]
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> ~[?:1.8.0_102]
> at
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> {code}
> * after that some NoSuchObjectExceptions follow
> * and then some replications seems to happen
> I don't fully understand this; I'll attach the logs...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-22626) Fix Replication related tests
[
https://issues.apache.org/jira/browse/HIVE-22626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aasha Medhi resolved HIVE-22626.
Resolution: Fixed
+1
Committed to master.
Thank you for the patch [~ayushtkn] and the review [~^sharma]
> Fix Replication related tests
> -
>
> Key: HIVE-22626
> URL: https://issues.apache.org/jira/browse/HIVE-22626
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Attachments: qalogs.tgz
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> For TestStatsReplicationScenariosACIDNoAutogather:
> this test is running "alone" because but still; it sometimes runs more than
> 40m which results in a timeout
> a jira search reveals that was pretty common:
>
> [https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc]
> from the hive logs:
> * it seems like after a few minutes this test starts there is an exception:
> {code:java}
> 2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient:
> Unable to shutdown metastore client. Will try closing transport directly.
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Socket closed
> at
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
> ~[libfb303-0.9.3.jar:?]
> at
> com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
> ~[libfb303-0.9.3.jar:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_102]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy62.close(Unknown Source) [?:?]
> at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:542)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.metadata.Hive.finalize(Hive.java:514)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.lang.System$2.invokeFinalize(System.java:1270) [?:1.8.0_102]
> at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
> [?:1.8.0_102]
> at java.lang.ref.Finalizer.access$100(Finalizer.java:34) [?:1.8.0_102]
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
> [?:1.8.0_102]
> Caused by: java.net.SocketException: Socket closed
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
> ~[?:1.8.0_102]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[?:1.8.0_102]
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> ~[?:1.8.0_102]
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> ~[?:1.8.0_102]
> at
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> {code}
> * after that some NoSuchObjectExceptions follow
> * and then some replications seems to happen
> I don't fully understand this; I'll attach the logs...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25336) Use single call to get tables in DropDatabaseAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-25336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383186#comment-17383186 ] Aasha Medhi commented on HIVE-25336: +1 committed to master. Thank you for the patch [~ayushtkn] and the review [~mgergely] > Use single call to get tables in DropDatabaseAnalyzer > - > > Key: HIVE-25336 > URL: https://issues.apache.org/jira/browse/HIVE-25336 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Optimise > org.apache.hadoop.hive.ql.ddl.database.drop.DropDatabaseAnalyzer.analyzeInternal(DropDatabaseAnalyzer.java:61), > where it fetches entire tables one by one. Move to a single call. This could > save around 20+ seconds when large number of tables are present. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25336) Use single call to get tables in DropDatabaseAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-25336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25336. Resolution: Fixed > Use single call to get tables in DropDatabaseAnalyzer > - > > Key: HIVE-25336 > URL: https://issues.apache.org/jira/browse/HIVE-25336 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Optimise > org.apache.hadoop.hive.ql.ddl.database.drop.DropDatabaseAnalyzer.analyzeInternal(DropDatabaseAnalyzer.java:61), > where it fetches entire tables one by one. Move to a single call. This could > save around 20+ seconds when large number of tables are present. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-23801) TestMiniLlapLocalCliDriver[replication_metrics_ingest] is flaky
[
https://issues.apache.org/jira/browse/HIVE-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aasha Medhi resolved HIVE-23801.
Resolution: Fixed
> TestMiniLlapLocalCliDriver[replication_metrics_ingest] is flaky
> ---
>
> Key: HIVE-23801
> URL: https://issues.apache.org/jira/browse/HIVE-23801
> Project: Hive
> Issue Type: Bug
> Components: repl
>Reporter: Peter Vary
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This test is flaky. See:
> [http://ci.hive.apache.org/job/hive-flaky-check/62/console]
> {code:java}
> 21:59:19 [INFO] ---
> 21:59:19 [INFO] T E S T S
> 21:59:19 [INFO] ---
> 21:59:19 [INFO] Running org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
> 22:01:56 [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 144.366 s <<< FAILURE! - in
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
> 22:01:56 [ERROR]
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[replication_metrics_ingest]
> Time elapsed: 124.174 s <<< FAILURE!
> 22:01:56 java.lang.AssertionError:
> 22:01:56 Client Execution succeeded but contained differences (error code =
> 1) after executing replication_metrics_ingest.q
> 22:01:56 76c76
> 22:01:56 < 3 repl2 1
> 22:01:56 ---
> 22:01:56 > 2 repl2 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23801) TestMiniLlapLocalCliDriver[replication_metrics_ingest] is flaky
[
https://issues.apache.org/jira/browse/HIVE-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377045#comment-17377045
]
Aasha Medhi commented on HIVE-23801:
+1 Committed to master. Thank you for the patch [~ayushtkn]
> TestMiniLlapLocalCliDriver[replication_metrics_ingest] is flaky
> ---
>
> Key: HIVE-23801
> URL: https://issues.apache.org/jira/browse/HIVE-23801
> Project: Hive
> Issue Type: Bug
> Components: repl
>Reporter: Peter Vary
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This test is flaky. See:
> [http://ci.hive.apache.org/job/hive-flaky-check/62/console]
> {code:java}
> 21:59:19 [INFO] ---
> 21:59:19 [INFO] T E S T S
> 21:59:19 [INFO] ---
> 21:59:19 [INFO] Running org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
> 22:01:56 [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 144.366 s <<< FAILURE! - in
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
> 22:01:56 [ERROR]
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[replication_metrics_ingest]
> Time elapsed: 124.174 s <<< FAILURE!
> 22:01:56 java.lang.AssertionError:
> 22:01:56 Client Execution succeeded but contained differences (error code =
> 1) after executing replication_metrics_ingest.q
> 22:01:56 76c76
> 22:01:56 < 3 repl2 1
> 22:01:56 ---
> 22:01:56 > 2 repl2 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25301) Expose notification log table through sys db
[ https://issues.apache.org/jira/browse/HIVE-25301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377040#comment-17377040 ] Aasha Medhi commented on HIVE-25301: +1 committed to master. Thank you for the patch [~ayushtkn] > Expose notification log table through sys db > > > Key: HIVE-25301 > URL: https://issues.apache.org/jira/browse/HIVE-25301 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Expose the notification_log table in RDBMS through Hive sys database -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25301) Expose notification log table through sys db
[ https://issues.apache.org/jira/browse/HIVE-25301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25301. Resolution: Fixed > Expose notification log table through sys db > > > Key: HIVE-25301 > URL: https://issues.apache.org/jira/browse/HIVE-25301 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Expose the notification_log table in RDBMS through Hive sys database -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25086) Create Ranger Deny Policy for replication db in all cases if hive.repl.ranger.target.deny.policy is set to true.
[ https://issues.apache.org/jira/browse/HIVE-25086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25086. Resolution: Fixed > Create Ranger Deny Policy for replication db in all cases if > hive.repl.ranger.target.deny.policy is set to true. > > > Key: HIVE-25086 > URL: https://issues.apache.org/jira/browse/HIVE-25086 > Project: Hive > Issue Type: Improvement >Reporter: Haymant Mangla >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25086) Create Ranger Deny Policy for replication db in all cases if hive.repl.ranger.target.deny.policy is set to true.
[ https://issues.apache.org/jira/browse/HIVE-25086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377037#comment-17377037 ] Aasha Medhi commented on HIVE-25086: +1 Committed to master. Thank you for the patch [~haymant] > Create Ranger Deny Policy for replication db in all cases if > hive.repl.ranger.target.deny.policy is set to true. > > > Key: HIVE-25086 > URL: https://issues.apache.org/jira/browse/HIVE-25086 > Project: Hive > Issue Type: Improvement >Reporter: Haymant Mangla >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25164) Execute Bootstrap REPL load DDL tasks in parallel
[ https://issues.apache.org/jira/browse/HIVE-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372133#comment-17372133 ] Aasha Medhi commented on HIVE-25164: +1 > Execute Bootstrap REPL load DDL tasks in parallel > - > > Key: HIVE-25164 > URL: https://issues.apache.org/jira/browse/HIVE-25164 > Project: Hive > Issue Type: Improvement >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25272) READ transactions are getting logged in NOTIFICATION LOG
[ https://issues.apache.org/jira/browse/HIVE-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371204#comment-17371204 ] Aasha Medhi commented on HIVE-25272: +1 > READ transactions are getting logged in NOTIFICATION LOG > > > Key: HIVE-25272 > URL: https://issues.apache.org/jira/browse/HIVE-25272 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > While READ transactions are already skipped from getting logged in > NOTIFICATION logs, few are still getting logged. Need to skip those > transactions as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25218) Add a replication migration validation tool for external tables
[ https://issues.apache.org/jira/browse/HIVE-25218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367910#comment-17367910 ] Aasha Medhi commented on HIVE-25218: Thank you for the patch [~ayushtkn] Committed to master. > Add a replication migration validation tool for external tables > --- > > Key: HIVE-25218 > URL: https://issues.apache.org/jira/browse/HIVE-25218 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Add a tool which can validate the migration of external tables post > replication from one cluster to another. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25231) Add an ability to migrate CSV generated to hive table in replstats
[ https://issues.apache.org/jira/browse/HIVE-25231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367909#comment-17367909 ] Aasha Medhi commented on HIVE-25231: Thank you for the patch [~ayushtkn] Committed to master > Add an ability to migrate CSV generated to hive table in replstats > -- > > Key: HIVE-25231 > URL: https://issues.apache.org/jira/browse/HIVE-25231 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Add an option to replstats.sh to load the CSV generated using the replication > policy into a hive table/view. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25207) Expose incremental load statistics via JMX
[ https://issues.apache.org/jira/browse/HIVE-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367907#comment-17367907 ] Aasha Medhi commented on HIVE-25207: Thank you for the patch [~ayushtkn] Committed to master > Expose incremental load statistics via JMX > -- > > Key: HIVE-25207 > URL: https://issues.apache.org/jira/browse/HIVE-25207 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Expose the incremental load details and statistics at per policy level in the > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25207) Expose incremental load statistics via JMX
[ https://issues.apache.org/jira/browse/HIVE-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25207. Resolution: Fixed > Expose incremental load statistics via JMX > -- > > Key: HIVE-25207 > URL: https://issues.apache.org/jira/browse/HIVE-25207 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Expose the incremental load details and statistics at per policy level in the > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25218) Add a replication migration validation tool for external tables
[ https://issues.apache.org/jira/browse/HIVE-25218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-25218: --- Description: Add a tool which can validate the migration of external tables post replication from one cluster to another. (was: Add a tool which can confirm migration of external tables post replication from one cluster to another.) > Add a replication migration validation tool for external tables > --- > > Key: HIVE-25218 > URL: https://issues.apache.org/jira/browse/HIVE-25218 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Add a tool which can validate the migration of external tables post > replication from one cluster to another. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25218) Add a replication migration validation tool for external tables
[ https://issues.apache.org/jira/browse/HIVE-25218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-25218: --- Summary: Add a replication migration validation tool for external tables (was: Add a replication migration tool for external tables) > Add a replication migration validation tool for external tables > --- > > Key: HIVE-25218 > URL: https://issues.apache.org/jira/browse/HIVE-25218 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Add a tool which can confirm migration of external tables post replication > from one cluster to another. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25165) Generate & track statistics per event type for incremental load in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25165. Resolution: Fixed > Generate & track statistics per event type for incremental load in > replication metrics > -- > > Key: HIVE-25165 > URL: https://issues.apache.org/jira/browse/HIVE-25165 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Generate and track statistics like mean, median. standard deviation, variance > etc per event type during incremental load and store them in replication > statistics -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25165) Generate & track statistics per event type for incremental load in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358051#comment-17358051 ] Aasha Medhi commented on HIVE-25165: +1 Committed to master. Thank you for the patch [~ayushtkn] > Generate & track statistics per event type for incremental load in > replication metrics > -- > > Key: HIVE-25165 > URL: https://issues.apache.org/jira/browse/HIVE-25165 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Generate and track statistics like mean, median. standard deviation, variance > etc per event type during incremental load and store them in replication > statistics -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25133) Allow custom configs for database level paths in external table replication
[ https://issues.apache.org/jira/browse/HIVE-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25133. Resolution: Fixed > Allow custom configs for database level paths in external table replication > --- > > Key: HIVE-25133 > URL: https://issues.apache.org/jira/browse/HIVE-25133 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Allow a way to provide configurations which should be used only by the > external data copy task of database level paths -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25133) Allow custom configs for database level paths in external table replication
[ https://issues.apache.org/jira/browse/HIVE-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358050#comment-17358050 ] Aasha Medhi commented on HIVE-25133: +1 Thank you for the patch [~ayushtkn] > Allow custom configs for database level paths in external table replication > --- > > Key: HIVE-25133 > URL: https://issues.apache.org/jira/browse/HIVE-25133 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Allow a way to provide configurations which should be used only by the > external data copy task of database level paths -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24924) Optimize checkpointing flow in incremental load
[ https://issues.apache.org/jira/browse/HIVE-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347301#comment-17347301 ] Aasha Medhi commented on HIVE-24924: +1 Committed to master. Thank you for the patch [~ayushtkn] > Optimize checkpointing flow in incremental load > --- > > Key: HIVE-24924 > URL: https://issues.apache.org/jira/browse/HIVE-24924 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Attempt reducing alter calls for checkpointing during repl load -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24924) Optimize checkpointing flow in incremental load
[ https://issues.apache.org/jira/browse/HIVE-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24924. Resolution: Fixed > Optimize checkpointing flow in incremental load > --- > > Key: HIVE-24924 > URL: https://issues.apache.org/jira/browse/HIVE-24924 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Attempt reducing alter calls for checkpointing during repl load -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24852) Add support for Snapshots during external table replication
[ https://issues.apache.org/jira/browse/HIVE-24852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24852. Resolution: Fixed > Add support for Snapshots during external table replication > --- > > Key: HIVE-24852 > URL: https://issues.apache.org/jira/browse/HIVE-24852 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Labels: pull-request-available > Attachments: Design Doc HDFS Snapshots for External Table > Replication-01.pdf > > Time Spent: 7h > Remaining Estimate: 0h > > Add support for use of snapshot diff for external table replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24852) Add support for Snapshots during external table replication
[ https://issues.apache.org/jira/browse/HIVE-24852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347034#comment-17347034 ] Aasha Medhi commented on HIVE-24852: +1 Committed to master. Thank you for the patch [~ayushtkn] > Add support for Snapshots during external table replication > --- > > Key: HIVE-24852 > URL: https://issues.apache.org/jira/browse/HIVE-24852 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Labels: pull-request-available > Attachments: Design Doc HDFS Snapshots for External Table > Replication-01.pdf > > Time Spent: 7h > Remaining Estimate: 0h > > Add support for use of snapshot diff for external table replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25092) Add a shell script to fetch the statistics of replication data copy tasks
[ https://issues.apache.org/jira/browse/HIVE-25092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345029#comment-17345029 ] Aasha Medhi commented on HIVE-25092: +1 Committed to master. Thank you for the patch [~ayushtkn] > Add a shell script to fetch the statistics of replication data copy tasks > - > > Key: HIVE-25092 > URL: https://issues.apache.org/jira/browse/HIVE-25092 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Add a shell script which can fetch the statistics of the Mapred(Distcp) jobs > launched as part of replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25092) Add a shell script to fetch the statistics of replication data copy tasks
[ https://issues.apache.org/jira/browse/HIVE-25092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25092. Resolution: Fixed > Add a shell script to fetch the statistics of replication data copy tasks > - > > Key: HIVE-25092 > URL: https://issues.apache.org/jira/browse/HIVE-25092 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Add a shell script which can fetch the statistics of the Mapred(Distcp) jobs > launched as part of replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25059) Alter event is converted to rename during replication
[ https://issues.apache.org/jira/browse/HIVE-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25059. Resolution: Fixed > Alter event is converted to rename during replication > - > > Key: HIVE-25059 > URL: https://issues.apache.org/jira/browse/HIVE-25059 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > In case the database/table name have different cases, while creating an alter > event it considers change of name and creates a RENAME event rather than ALTER -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25059) Alter event is converted to rename during replication
[ https://issues.apache.org/jira/browse/HIVE-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335279#comment-17335279 ] Aasha Medhi commented on HIVE-25059: +1 Committed to master. Thank you for the patch [~ayushtkn] > Alter event is converted to rename during replication > - > > Key: HIVE-25059 > URL: https://issues.apache.org/jira/browse/HIVE-25059 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > In case the database/table name have different cases, while creating an alter > event it considers change of name and creates a RENAME event rather than ALTER -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25035) Allow creating single copy tasks for configured paths during external table replication
[ https://issues.apache.org/jira/browse/HIVE-25035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25035. Resolution: Fixed > Allow creating single copy tasks for configured paths during external table > replication > --- > > Key: HIVE-25035 > URL: https://issues.apache.org/jira/browse/HIVE-25035 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > As of now one tasks per table is created for external table replication, in > case there are multiple tables under one common directory, provide a way to > create a single task for all those table. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25035) Allow creating single copy tasks for configured paths during external table replication
[ https://issues.apache.org/jira/browse/HIVE-25035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17331739#comment-17331739 ] Aasha Medhi commented on HIVE-25035: +1 committed to master. Thank you for the patch [~ayushtkn] > Allow creating single copy tasks for configured paths during external table > replication > --- > > Key: HIVE-25035 > URL: https://issues.apache.org/jira/browse/HIVE-25035 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > As of now one tasks per table is created for external table replication, in > case there are multiple tables under one common directory, provide a way to > create a single task for all those table. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24978) Optimise number of DROP_PARTITION events created.
[ https://issues.apache.org/jira/browse/HIVE-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24978. Resolution: Fixed > Optimise number of DROP_PARTITION events created. > - > > Key: HIVE-24978 > URL: https://issues.apache.org/jira/browse/HIVE-24978 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Even for drop partition with batches, presently there is one event for every > partition, optimise to merge them, to save the number of calls to HMS -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24978) Optimise number of DROP_PARTITION events created.
[ https://issues.apache.org/jira/browse/HIVE-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326284#comment-17326284 ] Aasha Medhi commented on HIVE-24978: +1 Committed to master. Thank you for the patch [~ayushtkn] > Optimise number of DROP_PARTITION events created. > - > > Key: HIVE-24978 > URL: https://issues.apache.org/jira/browse/HIVE-24978 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Even for drop partition with batches, presently there is one event for every > partition, optimise to merge them, to save the number of calls to HMS -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25002) modify condition for target of replication in statsUpdaterThread and PartitionManagementTask
[ https://issues.apache.org/jira/browse/HIVE-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326271#comment-17326271 ] Aasha Medhi commented on HIVE-25002: +1 Committed to master. Thank you for the patch [~haymant] > modify condition for target of replication in statsUpdaterThread and > PartitionManagementTask > > > Key: HIVE-25002 > URL: https://issues.apache.org/jira/browse/HIVE-25002 > Project: Hive > Issue Type: Improvement >Reporter: Haymant Mangla >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25002) modify condition for target of replication in statsUpdaterThread and PartitionManagementTask
[ https://issues.apache.org/jira/browse/HIVE-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25002. Resolution: Fixed > modify condition for target of replication in statsUpdaterThread and > PartitionManagementTask > > > Key: HIVE-25002 > URL: https://issues.apache.org/jira/browse/HIVE-25002 > Project: Hive > Issue Type: Improvement >Reporter: Haymant Mangla >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25016) Error while running repl dump with with db regex
[ https://issues.apache.org/jira/browse/HIVE-25016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-25016. Resolution: Fixed > Error while running repl dump with with db regex > > > Key: HIVE-25016 > URL: https://issues.apache.org/jira/browse/HIVE-25016 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Doing incremental dump with create-database event with dbRegex `*` gives the > following exception : > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: Execution Error, return code -101 from > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask. > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.EximUtil.updateIfCustomDbLocations(EximUtil.java:388) > at > org.apache.hadoop.hive.ql.parse.EximUtil.createDbExportDump(EximUtil.java:357) > at > org.apache.hadoop.hive.ql.parse.repl.dump.events.CreateDatabaseHandler.handle(CreateDatabaseHandler.java:42) > at > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.dumpEvent(ReplDumpTask.java:827) > at > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.incrementalDump(ReplDumpTask.java:632) > at > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.execute(ReplDumpTask.java:209) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) > at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:357) > at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330) > at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25016) Error while running repl dump with with db regex
[ https://issues.apache.org/jira/browse/HIVE-25016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324733#comment-17324733 ] Aasha Medhi commented on HIVE-25016: +1 Committed to master. Thank you for the patch [~^sharma] > Error while running repl dump with with db regex > > > Key: HIVE-25016 > URL: https://issues.apache.org/jira/browse/HIVE-25016 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Doing incremental dump with create-database event with dbRegex `*` gives the > following exception : > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: Execution Error, return code -101 from > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask. > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.EximUtil.updateIfCustomDbLocations(EximUtil.java:388) > at > org.apache.hadoop.hive.ql.parse.EximUtil.createDbExportDump(EximUtil.java:357) > at > org.apache.hadoop.hive.ql.parse.repl.dump.events.CreateDatabaseHandler.handle(CreateDatabaseHandler.java:42) > at > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.dumpEvent(ReplDumpTask.java:827) > at > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.incrementalDump(ReplDumpTask.java:632) > at > org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.execute(ReplDumpTask.java:209) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) > at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:357) > at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330) > at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24978) Optimise number of DROP_PARTITION events created.
[ https://issues.apache.org/jira/browse/HIVE-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-24978: --- Description: Even for drop partition with batches, presently there is one event for every partition, optimise to merge them, to save the number of calls to HMS (was: Presently there is one event for every drop, optimise to merge them, to save the number of calls to HMS) > Optimise number of DROP_PARTITION events created. > - > > Key: HIVE-24978 > URL: https://issues.apache.org/jira/browse/HIVE-24978 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Even for drop partition with batches, presently there is one event for every > partition, optimise to merge them, to save the number of calls to HMS -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24961) Use database name as default in the mapred job name for replication
[ https://issues.apache.org/jira/browse/HIVE-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24961. Resolution: Fixed > Use database name as default in the mapred job name for replication > --- > > Key: HIVE-24961 > URL: https://issues.apache.org/jira/browse/HIVE-24961 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Add database as job name for replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24961) Use database name as default in the mapred job name for replication
[ https://issues.apache.org/jira/browse/HIVE-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17316873#comment-17316873 ] Aasha Medhi commented on HIVE-24961: +1, Committed to master. Thank you for the patch [~ayushtkn] > Use database name as default in the mapred job name for replication > --- > > Key: HIVE-24961 > URL: https://issues.apache.org/jira/browse/HIVE-24961 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Add database as job name for replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24895) Add a DataCopyEnd stage in ReplStateLogTask for external table replication
[ https://issues.apache.org/jira/browse/HIVE-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24895. Resolution: Fixed > Add a DataCopyEnd stage in ReplStateLogTask for external table replication > -- > > Key: HIVE-24895 > URL: https://issues.apache.org/jira/browse/HIVE-24895 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Add a task to mark the end of external table copy. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24895) Add a DataCopyEnd stage in ReplStateLogTask for external table replication
[ https://issues.apache.org/jira/browse/HIVE-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17315432#comment-17315432 ] Aasha Medhi commented on HIVE-24895: +1 Committed to master. Thank you for the patch [~ayushtkn] and review [~pkumarsinha] > Add a DataCopyEnd stage in ReplStateLogTask for external table replication > -- > > Key: HIVE-24895 > URL: https://issues.apache.org/jira/browse/HIVE-24895 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Add a task to mark the end of external table copy. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24953) Set repl.source.for property in the db if db is under replication in incremental dump.
[ https://issues.apache.org/jira/browse/HIVE-24953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24953. Resolution: Fixed > Set repl.source.for property in the db if db is under replication in > incremental dump. > -- > > Key: HIVE-24953 > URL: https://issues.apache.org/jira/browse/HIVE-24953 > Project: Hive > Issue Type: Bug >Reporter: Haymant Mangla >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24953) Set repl.source.for property in the db if db is under replication in incremental dump.
[ https://issues.apache.org/jira/browse/HIVE-24953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17315429#comment-17315429 ] Aasha Medhi commented on HIVE-24953: Committed to master. Thank you for the patch [~haymant] > Set repl.source.for property in the db if db is under replication in > incremental dump. > -- > > Key: HIVE-24953 > URL: https://issues.apache.org/jira/browse/HIVE-24953 > Project: Hive > Issue Type: Bug >Reporter: Haymant Mangla >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24953) Set repl.source.for property in the db if db is under replication in incremental dump.
[ https://issues.apache.org/jira/browse/HIVE-24953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17315427#comment-17315427 ] Aasha Medhi commented on HIVE-24953: +1 > Set repl.source.for property in the db if db is under replication in > incremental dump. > -- > > Key: HIVE-24953 > URL: https://issues.apache.org/jira/browse/HIVE-24953 > Project: Hive > Issue Type: Bug >Reporter: Haymant Mangla >Assignee: Haymant Mangla >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24919) Optimise AddPartition replication call
[ https://issues.apache.org/jira/browse/HIVE-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307683#comment-17307683 ] Aasha Medhi commented on HIVE-24919: Committed to master. Thank you for the patch [~ayushtkn] > Optimise AddPartition replication call > -- > > Key: HIVE-24919 > URL: https://issues.apache.org/jira/browse/HIVE-24919 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Avoid unnecessary alter partition call while replaying add partition call > during replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24919) Optimise AddPartition replication call
[ https://issues.apache.org/jira/browse/HIVE-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307682#comment-17307682 ] Aasha Medhi commented on HIVE-24919: +1 > Optimise AddPartition replication call > -- > > Key: HIVE-24919 > URL: https://issues.apache.org/jira/browse/HIVE-24919 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Avoid unnecessary alter partition call while replaying add partition call > during replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24919) Optimise AddPartition replication call
[ https://issues.apache.org/jira/browse/HIVE-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24919. Resolution: Fixed > Optimise AddPartition replication call > -- > > Key: HIVE-24919 > URL: https://issues.apache.org/jira/browse/HIVE-24919 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Avoid unnecessary alter partition call while replaying add partition call > during replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24896) External table having same name as dropped managed table fails to replicate
[ https://issues.apache.org/jira/browse/HIVE-24896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305917#comment-17305917 ] Aasha Medhi commented on HIVE-24896: +1 > External table having same name as dropped managed table fails to replicate > --- > > Key: HIVE-24896 > URL: https://issues.apache.org/jira/browse/HIVE-24896 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24836) Add replication policy name and schedule id as a job name for all the distcp jobs
[ https://issues.apache.org/jira/browse/HIVE-24836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17297949#comment-17297949 ] Aasha Medhi commented on HIVE-24836: Committed to master. Thank you for the patch [~ayushtkn] > Add replication policy name and schedule id as a job name for all the distcp > jobs > - > > Key: HIVE-24836 > URL: https://issues.apache.org/jira/browse/HIVE-24836 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Add replication policy name and schedule id as a job name for all the distcp > jobs launched as part of the schedule > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24836) Add replication policy name and schedule id as a job name for all the distcp jobs
[ https://issues.apache.org/jira/browse/HIVE-24836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24836. Resolution: Fixed > Add replication policy name and schedule id as a job name for all the distcp > jobs > - > > Key: HIVE-24836 > URL: https://issues.apache.org/jira/browse/HIVE-24836 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Add replication policy name and schedule id as a job name for all the distcp > jobs launched as part of the schedule > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24836) Add replication policy name and schedule id as a job name for all the distcp jobs
[ https://issues.apache.org/jira/browse/HIVE-24836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17297947#comment-17297947 ] Aasha Medhi commented on HIVE-24836: +1 > Add replication policy name and schedule id as a job name for all the distcp > jobs > - > > Key: HIVE-24836 > URL: https://issues.apache.org/jira/browse/HIVE-24836 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Add replication policy name and schedule id as a job name for all the distcp > jobs launched as part of the schedule > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24624) Repl Load should detect the compatible staging dir
[ https://issues.apache.org/jira/browse/HIVE-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17293050#comment-17293050 ] Aasha Medhi commented on HIVE-24624: Reopened as the patch is not backward compatible and using this approach the issue is not solvable. Will need to probe at the dir structure to determine if the dump is compatible or not. > Repl Load should detect the compatible staging dir > -- > > Key: HIVE-24624 > URL: https://issues.apache.org/jira/browse/HIVE-24624 > Project: Hive > Issue Type: Improvement >Reporter: Pratyushotpal Madhukar >Assignee: Pratyushotpal Madhukar >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24624.patch > > Time Spent: 2h > Remaining Estimate: 0h > > Repl load in CDP when pointed to a staging dir should be able to detect > whether the staging dir has the dump structure in compatible format or not -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HIVE-24624) Repl Load should detect the compatible staging dir
[ https://issues.apache.org/jira/browse/HIVE-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi reopened HIVE-24624: > Repl Load should detect the compatible staging dir > -- > > Key: HIVE-24624 > URL: https://issues.apache.org/jira/browse/HIVE-24624 > Project: Hive > Issue Type: Improvement >Reporter: Pratyushotpal Madhukar >Assignee: Pratyushotpal Madhukar >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24624.patch > > Time Spent: 2h > Remaining Estimate: 0h > > Repl load in CDP when pointed to a staging dir should be able to detect > whether the staging dir has the dump structure in compatible format or not -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24733) Handle replication when db location and managed location is set to custom location on source
[
https://issues.apache.org/jira/browse/HIVE-24733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aasha Medhi updated HIVE-24733:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Handle replication when db location and managed location is set to custom
> location on source
>
>
> Key: HIVE-24733
> URL: https://issues.apache.org/jira/browse/HIVE-24733
> Project: Hive
> Issue Type: Task
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> {color:#172b4d} {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24733) Handle replication when db location and managed location is set to custom location on source
[
https://issues.apache.org/jira/browse/HIVE-24733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286095#comment-17286095
]
Aasha Medhi commented on HIVE-24733:
+1
> Handle replication when db location and managed location is set to custom
> location on source
>
>
> Key: HIVE-24733
> URL: https://issues.apache.org/jira/browse/HIVE-24733
> Project: Hive
> Issue Type: Task
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> {color:#172b4d} {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-24744) Deletion of previous dump dir fails with NPE for ptests
[ https://issues.apache.org/jira/browse/HIVE-24744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285114#comment-17285114 ] Aasha Medhi edited comment on HIVE-24744 at 2/16/21, 9:44 AM: -- Committed to master. Thank you for the patch [[~^sharma]] was (Author: aasha): Committed to master. Thank you for the patch [~^sharma] > Deletion of previous dump dir fails with NPE for ptests > --- > > Key: HIVE-24744 > URL: https://issues.apache.org/jira/browse/HIVE-24744 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24744.01.patch, HIVE-24744.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24744) Deletion of previous dump dir fails with NPE for ptests
[ https://issues.apache.org/jira/browse/HIVE-24744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285114#comment-17285114 ] Aasha Medhi commented on HIVE-24744: Committed to master. Thank you for the patch [~^sharma] > Deletion of previous dump dir fails with NPE for ptests > --- > > Key: HIVE-24744 > URL: https://issues.apache.org/jira/browse/HIVE-24744 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24744.01.patch, HIVE-24744.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24744) Deletion of previous dump dir fails with NPE for ptests
[ https://issues.apache.org/jira/browse/HIVE-24744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24744. Resolution: Fixed > Deletion of previous dump dir fails with NPE for ptests > --- > > Key: HIVE-24744 > URL: https://issues.apache.org/jira/browse/HIVE-24744 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24744.01.patch, HIVE-24744.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24624) Repl Load should detect the compatible staging dir
[ https://issues.apache.org/jira/browse/HIVE-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24624. Resolution: Fixed > Repl Load should detect the compatible staging dir > -- > > Key: HIVE-24624 > URL: https://issues.apache.org/jira/browse/HIVE-24624 > Project: Hive > Issue Type: Improvement >Reporter: Pratyushotpal Madhukar >Assignee: Pratyushotpal Madhukar >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24624.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Repl load in CDP when pointed to a staging dir should be able to detect > whether the staging dir has the dump structure in compatible format or not -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24624) Repl Load should detect the compatible staging dir
[ https://issues.apache.org/jira/browse/HIVE-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284633#comment-17284633 ] Aasha Medhi commented on HIVE-24624: +1 Committed to master. Thank you for the patch [~pmadhukar] and the review [~pkumarsinha] > Repl Load should detect the compatible staging dir > -- > > Key: HIVE-24624 > URL: https://issues.apache.org/jira/browse/HIVE-24624 > Project: Hive > Issue Type: Improvement >Reporter: Pratyushotpal Madhukar >Assignee: Pratyushotpal Madhukar >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24624.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Repl load in CDP when pointed to a staging dir should be able to detect > whether the staging dir has the dump structure in compatible format or not -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24755) Recycle fails if the cm root isn't absolute
[ https://issues.apache.org/jira/browse/HIVE-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24755. Resolution: Fixed > Recycle fails if the cm root isn't absolute > --- > > Key: HIVE-24755 > URL: https://issues.apache.org/jira/browse/HIVE-24755 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24755) Recycle fails if the cm root isn't absolute
[ https://issues.apache.org/jira/browse/HIVE-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284630#comment-17284630 ] Aasha Medhi commented on HIVE-24755: +1. Committed to master. Thank you for the patch [~ayushtkn] > Recycle fails if the cm root isn't absolute > --- > > Key: HIVE-24755 > URL: https://issues.apache.org/jira/browse/HIVE-24755 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24744) Deletion of previous dump dir fails with NPE for ptests
[ https://issues.apache.org/jira/browse/HIVE-24744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282231#comment-17282231 ] Aasha Medhi commented on HIVE-24744: +1 > Deletion of previous dump dir fails with NPE for ptests > --- > > Key: HIVE-24744 > URL: https://issues.apache.org/jira/browse/HIVE-24744 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24744.01.patch, HIVE-24744.02.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24698) Sync ACL's for the table directory during external table replication.
[ https://issues.apache.org/jira/browse/HIVE-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24698. Resolution: Fixed > Sync ACL's for the table directory during external table replication. > - > > Key: HIVE-24698 > URL: https://issues.apache.org/jira/browse/HIVE-24698 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Set similar ACL's to destination table directory in case the source has ACL's > enabled or set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24698) Sync ACL's for the table directory during external table replication.
[ https://issues.apache.org/jira/browse/HIVE-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280965#comment-17280965 ] Aasha Medhi commented on HIVE-24698: +1 > Sync ACL's for the table directory during external table replication. > - > > Key: HIVE-24698 > URL: https://issues.apache.org/jira/browse/HIVE-24698 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Set similar ACL's to destination table directory in case the source has ACL's > enabled or set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24698) Sync ACL's for the table directory during external table replication.
[ https://issues.apache.org/jira/browse/HIVE-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280966#comment-17280966 ] Aasha Medhi commented on HIVE-24698: Committed to master. Thank you for the patch [~ayushtkn] > Sync ACL's for the table directory during external table replication. > - > > Key: HIVE-24698 > URL: https://issues.apache.org/jira/browse/HIVE-24698 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Set similar ACL's to destination table directory in case the source has ACL's > enabled or set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24654) Table level replication support for Atlas metadata
[ https://issues.apache.org/jira/browse/HIVE-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17274337#comment-17274337 ] Aasha Medhi commented on HIVE-24654: +1 > Table level replication support for Atlas metadata > -- > > Key: HIVE-24654 > URL: https://issues.apache.org/jira/browse/HIVE-24654 > Project: Hive > Issue Type: Task >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24654.01.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Covers mainly Atlas export API payload change required to support table level > replication -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24674) Set repl.source.for property in the db if db is under replication
[ https://issues.apache.org/jira/browse/HIVE-24674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24674. Resolution: Fixed > Set repl.source.for property in the db if db is under replication > - > > Key: HIVE-24674 > URL: https://issues.apache.org/jira/browse/HIVE-24674 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > Add repl.source.for property in the database in case not already set, if the > database is under replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24674) Set repl.source.for property in the db if db is under replication
[ https://issues.apache.org/jira/browse/HIVE-24674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17274315#comment-17274315 ] Aasha Medhi commented on HIVE-24674: Thank you for the patch [~ayushtkn] Committed to master. > Set repl.source.for property in the db if db is under replication > - > > Key: HIVE-24674 > URL: https://issues.apache.org/jira/browse/HIVE-24674 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > Add repl.source.for property in the database in case not already set, if the > database is under replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24674) Set repl.source.for property in the db if db is under replication
[ https://issues.apache.org/jira/browse/HIVE-24674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17274314#comment-17274314 ] Aasha Medhi commented on HIVE-24674: +1 > Set repl.source.for property in the db if db is under replication > - > > Key: HIVE-24674 > URL: https://issues.apache.org/jira/browse/HIVE-24674 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Add repl.source.for property in the database in case not already set, if the > database is under replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24675) Handle external table replication for HA with same NS and lazy copy.
[ https://issues.apache.org/jira/browse/HIVE-24675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24675. Resolution: Fixed > Handle external table replication for HA with same NS and lazy copy. > > > Key: HIVE-24675 > URL: https://issues.apache.org/jira/browse/HIVE-24675 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24675.01.patch, HIVE-24675.02.patch, > HIVE-24675.03.patch, HIVE-24675.04.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24675) Handle external table replication for HA with same NS and lazy copy.
[ https://issues.apache.org/jira/browse/HIVE-24675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17272868#comment-17272868 ] Aasha Medhi commented on HIVE-24675: Thank you for the patch [~^sharma] Committed to master. > Handle external table replication for HA with same NS and lazy copy. > > > Key: HIVE-24675 > URL: https://issues.apache.org/jira/browse/HIVE-24675 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24675.01.patch, HIVE-24675.02.patch, > HIVE-24675.03.patch, HIVE-24675.04.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24675) Handle external table replication for HA with same NS and lazy copy.
[ https://issues.apache.org/jira/browse/HIVE-24675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17272530#comment-17272530 ] Aasha Medhi commented on HIVE-24675: +1 > Handle external table replication for HA with same NS and lazy copy. > > > Key: HIVE-24675 > URL: https://issues.apache.org/jira/browse/HIVE-24675 > Project: Hive > Issue Type: Bug >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24675.01.patch, HIVE-24675.02.patch, > HIVE-24675.03.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24676) Show locks call failing for postgres RDBMS if the txnId filter is passed
[ https://issues.apache.org/jira/browse/HIVE-24676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17269757#comment-17269757 ] Aasha Medhi commented on HIVE-24676: Committed to master. Thank you for the review [~dkuzmenko] > Show locks call failing for postgres RDBMS if the txnId filter is passed > > > Key: HIVE-24676 > URL: https://issues.apache.org/jira/browse/HIVE-24676 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24676) Show locks call failing for postgres RDBMS if the txnId filter is passed
[ https://issues.apache.org/jira/browse/HIVE-24676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi resolved HIVE-24676. Resolution: Fixed > Show locks call failing for postgres RDBMS if the txnId filter is passed > > > Key: HIVE-24676 > URL: https://issues.apache.org/jira/browse/HIVE-24676 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-24676) Show locks call failing for postgres RDBMS if the txnId filter is passed
[ https://issues.apache.org/jira/browse/HIVE-24676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-24676 started by Aasha Medhi. -- > Show locks call failing for postgres RDBMS if the txnId filter is passed > > > Key: HIVE-24676 > URL: https://issues.apache.org/jira/browse/HIVE-24676 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24676) Show locks call failing for postgres RDBMS if the txnId filter is passed
[ https://issues.apache.org/jira/browse/HIVE-24676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-24676: --- Summary: Show locks call failing for postgres RDBMS if the txnId filter is passed (was: show locks call failing for postgres RDBMS if the txnId filter is passed) > Show locks call failing for postgres RDBMS if the txnId filter is passed > > > Key: HIVE-24676 > URL: https://issues.apache.org/jira/browse/HIVE-24676 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24676) show locks call failing for postgres RDBMS if the txnId filter is passed
[ https://issues.apache.org/jira/browse/HIVE-24676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi reassigned HIVE-24676: -- > show locks call failing for postgres RDBMS if the txnId filter is passed > > > Key: HIVE-24676 > URL: https://issues.apache.org/jira/browse/HIVE-24676 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-24558: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24558.01.patch, HIVE-24558.02.patch, > HIVE-24558.03.patch, HIVE-24558.04.patch, HIVE-24558.05.patch, > HIVE-24558.06.patch, HIVE-24558.07.patch, HIVE-24558.08.patch, > HIVE-24558.09.patch > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
