[
https://issues.apache.org/jira/browse/HIVE-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172972#comment-14172972
]
Thomas Friedrich commented on HIVE-7953:
----------------------------------------
To fix all dbtxnmgr_* tests, we need to add
<property>
<name>hive.in.test</name>
<value>true</value>
<description>Internal marker for test. Used for masking env-dependent
values</description>
</property>
in the hive-site.xml under data/conf/spark.
I don't think we need a specific one for Spark. Even though they added
HIVE_IN_TEZ, the hive-site.xml for tez also sets HIVE_IN_TEST and HIVE_IN_TEZ
is not used anywhere else besides in TxnHandler which also checks HIVE_IN_TEST
already.
In addition to setting this value, we need to generate the output files for the
test cases under /hive/ql/src/test/results/clientpositive/spark. When I re-ran
the tests, they were successful. I will open a follow-up JIRA and provide the
patches.
> Investigate query failures (2)
> ------------------------------
>
> Key: HIVE-7953
> URL: https://issues.apache.org/jira/browse/HIVE-7953
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Brock Noland
> Assignee: Thomas Friedrich
>
> I ran all q-file tests and the following failed with an exception:
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-SPARK-ALL-TESTS-Build/lastCompletedBuild/testReport/
> we don't necessary want to run all these tests as part of the spark tests,
> but we should understand why they failed with an exception. This JIRA is to
> look into these failures and document them with one of:
> * New JIRA
> * Covered under existing JIRA
> * More investigation required
> Tests:
> {noformat}
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_temp_table_external
> 0.33 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucket_num_reducers
> 4.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_2
> 11 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
> 0.65 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_4
> 4.7 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_7
> 2.8 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_2
> 5.5 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_position
> 1.5 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_18_part_external
> 2.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_6
> 11 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_smb_mapjoin_11
> 5.1 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_8
> 10 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parquet_join
> 5.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_empty_dyn_part
> 0.81 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact1
> 0.31 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_ddl1
> 0.26 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_query2
> 0.73 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_3
> 8.5 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_query5
> 0.34 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_rcfile_bigdata
> 0.93 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_multi_single_reducer
> 6.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact3
> 2.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact2
> 0.56 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_partscan_1_23
> 3.1 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_list_bucket_dml_2
> 4.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_15_external_part
> 3.2 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_16_part_external
> 2.8 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_17_part_managed
> 3.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_20_part_managed_location
> 3.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_19_00_part_external_location
> 6.9 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_external_table_with_space_in_location_path
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
