[
https://issues.apache.org/jira/browse/HIVE-24673?focusedWorklogId=542994&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-542994
]
ASF GitHub Bot logged work on HIVE-24673:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Jan/21 18:43
Start Date: 27/Jan/21 18:43
Worklog Time Spent: 10m
Work Description: mustafaiman commented on a change in pull request #1902:
URL: https://github.com/apache/hive/pull/1902#discussion_r565547563
##########
File path: ql/src/test/results/clientnegative/avro_non_nullable_union.q.out
##########
@@ -34,5 +34,39 @@ PREHOOK: query: INSERT OVERWRITE TABLE
union_non_nullable_test_avro SELECT * FRO
PREHOOK: type: QUERY
PREHOOK: Input: default@union_nullable_test_text
PREHOOK: Output: default@union_non_nullable_test_avro
+Status: Failed
+Vertex failed, vertexName=Map 1, vertexId=vertex_#ID#, diagnostics=[Task
failed, taskId=task_#ID#, diagnostics=[TaskAttempt 0 failed, info=[Error: Error
while running task ( failure ) : attempt_#ID#:java.lang.RuntimeException:
java.lang.RuntimeException: Hive Runtime Error while closing operators
#### A masked pattern was here ####
-FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
+Caused by: java.lang.RuntimeException: Hive Runtime Error while closing
operators
+#### A masked pattern was here ####
+Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Avro could not
validate record against schema (record = {"id": 3, "value": null}) (schema =
{"type":"record","name":"nullable","fields":[{"name":"id","type":"int"},{"name":"value","type":["int","double"]}]})
+#### A masked pattern was here ####
+Caused by: Avro could not validate record against schema (record = {"id": 3,
"value": null}) (schema =
{"type":"record","name":"nullable","fields":[{"name":"id","type":"int"},{"name":"value","type":["int","double"]}]})
Review comment:
It is not exactly that. The same stack trace is seen twice because
tez.am.task.max.failed.attempts is set to 2 in tez-site.xml. If you look
carefully you can see that first exception is because `TaskAttempt 0` failed
and the second one is because `TaskAttempt 1` failed. This is the case for all
the tests.
I thought about setting the config to 1 but I did not beause:
1. The config file is shared with positive tests and surely there are tests
depend on second attempt on both positive and negative tests side. This config
is not configurable in runtime. So we will probably need to introduce another
driver to separate tests that require two attempts and the ones that are ok
with one attempt.
2. Once you know Tez attempts tasks twice, the output does not look so messy
to me. It shows exactly what happened. Also, attempt count is almost never set
to 1 in production. So I think this is a better test than single-attempt test.
3. We can catch if the two attempts fail for different reasons somehow. Like
in a scenario where first attempt fails with the expected exception but the
second attempt goes rogue because we mess with something mutable in the first
run.
On the other hand, setting attempt count to 1 will reduce test time for
negative tests by roughly 50%.
I am leaning on keeping the attempt count at 2 and the output as verbose as
it is right now. What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 542994)
Time Spent: 1h 40m (was: 1.5h)
> Migrate NegativeCliDriver and NegativeMinimrCliDriver to llap
> -------------------------------------------------------------
>
> Key: HIVE-24673
> URL: https://issues.apache.org/jira/browse/HIVE-24673
> Project: Hive
> Issue Type: Improvement
> Reporter: Mustafa İman
> Assignee: Mustafa İman
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> These test drivers should run on llap. Otherwise we can run into situations
> where certain queries correctly fail on MapReduce but not on Tez.
> Also, it is better if negative cli drivers does not mask "Caused by" lines in
> test output. Otherwise, a query may start to fail for other reasons than the
> expected one and we do not realize it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
