[jira] [Created] (HIVE-23068) Error when submitting fragment to LLAP via external client: IllegalStateException: Only a single registration allowed per entity
Jason Dere created HIVE-23068:
-
Summary: Error when submitting fragment to LLAP via external
client: IllegalStateException: Only a single registration allowed per entity
Key: HIVE-23068
URL: https://issues.apache.org/jira/browse/HIVE-23068
Project: Hive
Issue Type: Bug
Components: llap
Reporter: Jason Dere
Assignee: Jason Dere
LLAP external client (via hive-warehouse-connector) somehow seems to be sending
duplicate submissions for the same fragment/attempt. When the 2nd request is
sent this results in the following error:
{noformat}
2020-03-17T06:49:11,239 WARN [IPC Server handler 2 on 15001 ()]
org.apache.hadoop.ipc.Server: IPC Server handler 2 on 15001, call Call#75
Retry#0 org.apache.hadoop.hive.llap.protocol.LlapProtocolBlockingPB.submitWork
from 19.40.252.114:33906
java.lang.IllegalStateException: Only a single registration allowed per entity.
Duplicate for TaskWrapper{task=attempt_1854104024183112753_6052_0_00_000128_1,
inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true,
canFinish=true, canFinish(in queue)=true, isGuaranteed=false,
firstAttemptStartTime=1584442003327, dagStartTime=1584442003327,
withinDagPriority=0, vertexParallelism= 2132, selfAndUpstreamParallelism= 2132,
selfAndUpstreamComplete= 0}
at
org.apache.hadoop.hive.llap.daemon.impl.QueryInfo$FinishableStateTracker.registerForUpdates(QueryInfo.java:233)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.QueryInfo.registerForFinishableStateUpdates(QueryInfo.java:205)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.QueryFragmentInfo.registerForFinishableStateUpdates(QueryFragmentInfo.java:160)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService$TaskWrapper.maybeRegisterForFinishedStateNotifications(TaskExecutorService.java:1167)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService.schedule(TaskExecutorService.java:564)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService.schedule(TaskExecutorService.java:93)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:292)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:610)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:122)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:22695)
~[hive-exec-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.32-1]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
~[hadoop-common-3.1.1.3.1.4.26-3.jar:?]
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
~[hadoop-common-3.1.1.3.1.4.26-3.jar:?]
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
~[hadoop-common-3.1.1.3.1.4.26-3.jar:?]
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
~[hadoop-common-3.1.1.3.1.4.26-3.jar:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_191]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_191]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
~[hadoop-common-3.1.1.3.1.4.26-3.jar:?]
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
~[hadoop-common-3.1.1.3.1.4.26-3.jar:?]
{noformat}
I think the issue here is that this error occurred too late - based on the
stack trace, LLAP has already accepted/registered the fragment. The subsequent
cleanup of this fragment/attempt also affects the first request. Which results
in the LLAP crash described in HIVE-23061:
{noformat}
2020-03-17T06:49:11,304 ERROR [ExecutionCompletionThread #0 ()]
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Thread
Thread[ExecutionCompletionThread #0,5,main] threw an Exception. Shutting down
now...
java.lang.IllegalStateException: Cannot invoke unregister on an entity which
has not been registered
at com.google.common.base.Preconditions.checkState(Preconditions.java:508)
~[hive-exec-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.32-1]
at
org.apache.hadoop.hive.llap.daemon.impl.QueryInfo$FinishableStateTracker.unregisterForUpdates(QueryInfo.java:256)
~[hive-llap-server-3.1.0.3.1.4.26-3.jar:3.1.0.3.1.4.26-3]
at
Resubmit Patch to re-generate failures?
Hi Hive Dev community! I’ve opened a Jira to propose a simple enhancement to hive.exec.stagingdir: https://issues.apache.org/jira/browse/HIVE-22928 The automatic QA reported checkstyle and license warnings, but the full report was garbage collected before I got back to look at it. I haven’t been able to reproduce the same warnings in my local build environment. Since then, I’ve tried to re-submit the patch to re-generate the warnings so I can fix them, but I haven’t been able to. I’ve tried stopping progress, resuming progress, marking the patch as available again, etc. Does anyone have any suggestions? Thanks! -- Tom Poepping Software Development Engineer II AWS EMR
[jira] [Created] (HIVE-23067) Use batch inserts in TxnHandler
Marton Bod created HIVE-23067: - Summary: Use batch inserts in TxnHandler Key: HIVE-23067 URL: https://issues.apache.org/jira/browse/HIVE-23067 Project: Hive Issue Type: Improvement Reporter: Marton Bod Assignee: Marton Bod To reduce the number of database calls and network roundtrips, we could use more batching in TxnHandler, where currently in many places we call insert commands in loops sequentially. Some examples: * openTxns (TXNS, REPL_TXN_MAP * commitTxn (COMPLETED_TXN_COMPONENTS) * replTableWriteIdState (TXN_TO_WRITE_ID) * allocateTableWriteIds (TXN_TO_WRITE_ID) * -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72234: HIVE-22785
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72234/ --- (Updated March 23, 2020, 3:30 p.m.) Review request for hive and Jesús Camacho Rodríguez. Bugs: HIVE-22785 https://issues.apache.org/jira/browse/HIVE-22785 Repository: hive-git Description --- Update/delete/merge statements not optimized through CBO Diffs (updated) - itests/hive-blobstore/src/test/results/clientpositive/map_join_on_filter.q.out 653faab00a itests/src/test/resources/testconfiguration.properties f71ed3d240 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelDistribution.java e5f4c8492e ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 04b3888a25 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelJson.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelJsonImpl.java 0d45eb0c61 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveSortExchange.java 880cae70f9 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortExchangeTransposeRule.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 871c411e70 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 53d68e872a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortExchangePullUpConstantsRule.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortLimitPullUpConstantsRule.java e51b2b6ebc ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortPullUpConstantsRuleBase.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java e03e96ff12 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java 31619c0314 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/opconventer/HiveSortExchangeVisitor.java 68227db1ee ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/opconventer/JoinVisitor.java 0286d54ea0 ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 6589eeb39b ql/src/java/org/apache/hadoop/hive/ql/parse/RewriteSemanticAnalyzer.java 31068cb8c3 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 841f92befc ql/src/test/queries/clientpositive/authorization_view_disable_cbo_1.q be50b69830 ql/src/test/queries/clientpositive/sort_acid.q PRE-CREATION ql/src/test/results/clientpositive/acid_view_delete.q.out 3771c3ba63 ql/src/test/results/clientpositive/authorization_view_disable_cbo_1.q.out b609982bb9 ql/src/test/results/clientpositive/auto_join0.q.out 665cf28dea ql/src/test/results/clientpositive/auto_join15.q.out 6ea9db28f0 ql/src/test/results/clientpositive/auto_join20.q.out 6bbcb47084 ql/src/test/results/clientpositive/auto_join21.q.out b0af07c93e ql/src/test/results/clientpositive/auto_join23.q.out 9dcfc1aa6f ql/src/test/results/clientpositive/auto_join28.q.out 6b27398f1e ql/src/test/results/clientpositive/auto_join29.q.out ade39bdc1a ql/src/test/results/clientpositive/auto_join31.q.out 73651d36f9 ql/src/test/results/clientpositive/cbo_rp_auto_join0.q.out 72e041e767 ql/src/test/results/clientpositive/correlationoptimizer14.q.out b8d764841b ql/src/test/results/clientpositive/ctas.q.out 23cc8b967e ql/src/test/results/clientpositive/ctas_char.q.out e0fc050876 ql/src/test/results/clientpositive/ctas_date.q.out 44f112651c ql/src/test/results/clientpositive/ctas_varchar.q.out d9a9519fbf ql/src/test/results/clientpositive/identity_project_remove_skip.q.out 1176c7c0d4 ql/src/test/results/clientpositive/input4_limit.q.out 34f4821c7d ql/src/test/results/clientpositive/input_part7.q.out 15a388e88a ql/src/test/results/clientpositive/join0.q.out cda0f3f0a5 ql/src/test/results/clientpositive/join15.q.out 664f2e0614 ql/src/test/results/clientpositive/join20.q.out 9327cf453a ql/src/test/results/clientpositive/join21.q.out d25eb5cb09 ql/src/test/results/clientpositive/join23.q.out 4f8f0d8c14 ql/src/test/results/clientpositive/join40.q.out b0942ea3e7 ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 699398bc9d ql/src/test/results/clientpositive/llap/acid_vectorization_original.q.out 2e43110b35 ql/src/test/results/clientpositive/llap/auto_join0.q.out 31b776fd5d ql/src/test/results/clientpositive/llap/auto_join21.q.out df866d8ce6 ql/src/test/results/clientpositive/llap/auto_join29.q.out 9c3a0b1388 ql/src/test/results/clientpositive/llap/auto_join30.q.out d81dadb799 ql/src/test/results/clientpositive/llap/check_constraint.q.out b4acc55f15 ql/src/test/results/clientpositive/llap/ctas.q.out 275bebd3b9
Re: Review Request 72234: HIVE-22785
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72234/ --- (Updated March 23, 2020, 7:01 a.m.) Review request for hive and Jesús Camacho Rodríguez. Bugs: HIVE-22785 https://issues.apache.org/jira/browse/HIVE-22785 Repository: hive-git Description --- Update/delete/merge statements not optimized through CBO Diffs (updated) - itests/hive-blobstore/src/test/results/clientpositive/map_join_on_filter.q.out 653faab00a itests/src/test/resources/testconfiguration.properties f71ed3d240 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 04b3888a25 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelJson.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelJsonImpl.java 0d45eb0c61 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveSortExchange.java 880cae70f9 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortExchangeTransposeRule.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 53d68e872a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortExchangePullUpConstantsRule.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java e03e96ff12 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java 31619c0314 ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 6589eeb39b ql/src/java/org/apache/hadoop/hive/ql/parse/RewriteSemanticAnalyzer.java 31068cb8c3 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 841f92befc ql/src/test/queries/clientpositive/authorization_view_disable_cbo_1.q be50b69830 ql/src/test/queries/clientpositive/sort_acid.q PRE-CREATION ql/src/test/results/clientpositive/acid_view_delete.q.out 3771c3ba63 ql/src/test/results/clientpositive/authorization_view_disable_cbo_1.q.out b609982bb9 ql/src/test/results/clientpositive/auto_join0.q.out 665cf28dea ql/src/test/results/clientpositive/auto_join15.q.out 6ea9db28f0 ql/src/test/results/clientpositive/auto_join20.q.out 6bbcb47084 ql/src/test/results/clientpositive/auto_join21.q.out b0af07c93e ql/src/test/results/clientpositive/auto_join23.q.out 9dcfc1aa6f ql/src/test/results/clientpositive/auto_join28.q.out 6b27398f1e ql/src/test/results/clientpositive/auto_join29.q.out ade39bdc1a ql/src/test/results/clientpositive/auto_join31.q.out 73651d36f9 ql/src/test/results/clientpositive/cbo_rp_auto_join0.q.out 72e041e767 ql/src/test/results/clientpositive/correlationoptimizer14.q.out b8d764841b ql/src/test/results/clientpositive/ctas.q.out 23cc8b967e ql/src/test/results/clientpositive/ctas_char.q.out e0fc050876 ql/src/test/results/clientpositive/ctas_date.q.out 44f112651c ql/src/test/results/clientpositive/ctas_varchar.q.out d9a9519fbf ql/src/test/results/clientpositive/identity_project_remove_skip.q.out 1176c7c0d4 ql/src/test/results/clientpositive/input4_limit.q.out 34f4821c7d ql/src/test/results/clientpositive/input_part7.q.out 15a388e88a ql/src/test/results/clientpositive/join0.q.out cda0f3f0a5 ql/src/test/results/clientpositive/join15.q.out 664f2e0614 ql/src/test/results/clientpositive/join20.q.out 9327cf453a ql/src/test/results/clientpositive/join21.q.out d25eb5cb09 ql/src/test/results/clientpositive/join23.q.out 4f8f0d8c14 ql/src/test/results/clientpositive/join40.q.out b0942ea3e7 ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 699398bc9d ql/src/test/results/clientpositive/llap/acid_vectorization_original.q.out 2e43110b35 ql/src/test/results/clientpositive/llap/auto_join0.q.out 31b776fd5d ql/src/test/results/clientpositive/llap/auto_join21.q.out df866d8ce6 ql/src/test/results/clientpositive/llap/auto_join29.q.out 9c3a0b1388 ql/src/test/results/clientpositive/llap/auto_join30.q.out d81dadb799 ql/src/test/results/clientpositive/llap/check_constraint.q.out b4acc55f15 ql/src/test/results/clientpositive/llap/ctas.q.out 275bebd3b9 ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_3.q.out 2522677d95 ql/src/test/results/clientpositive/llap/dynpart_sort_optimization_acid.q.out 57dbbe34c3 ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 74d0190dfe ql/src/test/results/clientpositive/llap/explainuser_1.q.out b82a055079 ql/src/test/results/clientpositive/llap/identity_project_remove_skip.q.out a8a5e38602 ql/src/test/results/clientpositive/llap/insert_into_default_keyword.q.out bdf1a65cb1 ql/src/test/results/clientpositive/llap/join0.q.out 0a81c5bca3 ql/src/test/results/clientpositive/llap/llap_acid.q.out b8a4058f6c
