[
https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674236#comment-16674236
]
Hive QA commented on HIVE-20853:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946763/HIVE-20853.1.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 75 failed/errored test(s), 14676 tests
executed
*Failed tests:*
{noformat}
TestLlapDaemonUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=332)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=196)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=183)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testChangeGuaranteedTotal
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testConcurrentUpdateWithError
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testConcurrentUpdates
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testConcurrentUpdatesBeforeMessage
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityDelayedAllocation
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityFallbackToNonLocal
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorDelayedAllocation
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedQueeTaskSelectionAfterScheduled
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForceLocalityTest1
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityMultiplePreemptionsSameHost1
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityMultiplePreemptionsSameHost2
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityNotInDelayedQueue
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityPreemption
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityUnknownHost
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testGuaranteedScheduling
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testGuaranteedTransfer
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testHeartbeatInconsistency
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testHostPreferenceMissesConsistentPartialAlive
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testHostPreferenceMissesConsistentRollover
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testHostPreferenceUnknownAndNotSpecified
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testNoForceLocalityCounterTest1
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testNoLocalityNotInDelayedQueue
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testNodeDisabled
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testNodeReEnabled
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testPreemption
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testPreemptionChoiceTimeOrdering
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testSimpleLocalAllocation
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testSimpleNoLocalityAllocation
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testUpdateOnFinishingTask
(batchId=278)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testUpdateWithError
(batchId=278)
org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits.org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits
(batchId=256)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testComplexQuery (batchId=258)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=258)
org.apache.hive.jdbc.TestJdbcWithMiniLlapRow.testComplexQuery (batchId=258)
org.apache.hive.jdbc.TestJdbcWithMiniLlapVectorArrow.testComplexQuery
(batchId=261)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.org.apache.hive.jdbc.TestTriggersMoveWorkloadManager
(batchId=258)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.org.apache.hive.jdbc.TestTriggersNoTezSessionPool
(batchId=258)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.org.apache.hive.jdbc.TestTriggersTezSessionPoolManager
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14733/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14733/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14733/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 75 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946763 - PreCommit-HIVE-Build
> Expose ShuffleHandler.registerDag in the llap daemon API
> --------------------------------------------------------
>
> Key: HIVE-20853
> URL: https://issues.apache.org/jira/browse/HIVE-20853
> Project: Hive
> Issue Type: Improvement
> Components: llap
> Affects Versions: 3.1.0
> Reporter: Jaume M
> Assignee: Jaume M
> Priority: Critical
> Attachments: HIVE-20853.1.patch, HIVE-20853.2.patch
>
>
> Currently DAGs are only registered when a submitWork is called for that DAG.
> At this point the crendentials are added to the ShuffleHandler and it can
> start serving.
> However Tez might (and will) schedule tasks to fetch from the ShuffleHandler
> before anything of this happens and all this tasks will fail which may
> results in the query failing.
> This happens in the scenario in which a LlapDaemon just comes up and tez
> fetchers try to open a connection before a DAG has been registered.
> Adding this API will allow to register the DAG against the Daemon when the AM
> notices that a new Daemon is up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
