[
https://issues.apache.org/jira/browse/HIVE-16666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015496#comment-16015496
]
Hive QA commented on HIVE-16666:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12868659/HIVE-16666.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 10730 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_blobstore]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_hdfs_to_blobstore]
(batchId=239)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[crtseltbl_serdeprops]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_uses_table_location]
(batchId=1)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_auto_purge_tables]
(batchId=162)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_ctas]
(batchId=162)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_drop_partition]
(batchId=162)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_drop_table]
(batchId=163)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_partition_dynamic]
(batchId=164)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_partition_static]
(batchId=161)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_values]
(batchId=163)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_unencrypted_tbl]
(batchId=165)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_load_data_to_encrypted_tables]
(batchId=162)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=162)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_select_read_only_encrypted_tbl]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=97)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[ctas_noemptyfolder]
(batchId=88)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=287)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5322/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5322/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5322/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12868659 - PreCommit-HIVE-Build
> Set hive.exec.stagingdir a relative directory or a sub directory of
> distination data directory will cause Hive to delete the intermediate query
> results
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-16666
> URL: https://issues.apache.org/jira/browse/HIVE-16666
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 3.0.0
> Reporter: yangfang
> Assignee: yangfang
> Priority: Critical
> Attachments: HIVE-16666.1.patch
>
>
> Set hive.exec.stagingdir=./*, for example set hive.exec.stagingdir=./opq8.
> Then excute a query like this:
> insert overwrite table test2 select * from test3;
> You will get the error like this:
> hive> set hive.exec.stagingdir=./opq8;
> hive> insert overwrite table test2 select * from test3;
> Query ID = mr_20170515134831_28ee392d-0d5a-4e47-b80c-dfcd31691b02
> Total jobs = 3
> Launching Job 1 out of 3
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1494818119523_0008, Tracking URL =
> http://zdh77:8088/proxy/application_1494818119523_0008/
> Kill Command = /opt/ZDH/parcels/lib/hadoop/bin/hadoop job -kill
> job_1494818119523_0008
> Hadoop job information for Stage-1: number of mappers: 0; number of reducers: > 0
> 2017-05-15 13:48:51,487 Stage-1 map = 0%, reduce = 0%
> Ended Job = job_1494818119523_0008
> Stage-3 is selected by condition resolver.
> Stage-2 is filtered out by condition resolver.
> Stage-4 is filtered out by condition resolver.
> Moving data to directory
> hdfs://nameservice/hive/test2/opqt8_hive_2017-05-15_13-48-31_558_6151032330134038151-1/-ext-10000
> Loading data to table default.test2
> Moved:
> 'hdfs://nameservice/hive/test2/opqt8_hive_2017-05-15_13-48-31_558_6151032330134038151-1'
> to trash at: hdfs://nameservice/user/mr/.Trash/Current
> Failed with exception Unable to move source
> hdfs://nameservice/hive/test2/opqt8_hive_2017-05-15_13-48-31_558_6151032330134038151-1/-ext-10000
> to destination hdfs://nameservice/hive/test2
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
> hdfs://nameservice/hive/test2/opqt8_hive_2017-05-15_13-48-31_558_6151032330134038151-1/-ext-10000
> to destination hdfs://nameservice/hive/test2
> MapReduce Jobs Launched:
> Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 SUCCESS
> Total MapReduce CPU Time Spent: 0 msec
> hive>
> hive.exec.stagingdir=./opq8 is a relative path for destination write
> directory /hive/test2. Hive will create a temporary directory
> /hive/test2/opq8_hive* for intermediate query results. Later in the move
> staging, Hive will delete or trash the sub directory under the /hive/test2
> who's name does not begin with "_" or "." in order to move data to this
> directory. You can see its processing logic in
> org.apache.hadoop.hive.ql.metadata.trashFilesUnderDir.
> My modification method is: if stagingdir is a sub directory of the
> destination write directory. I add a "." in front of stagingdir. now
> temporary directory will be /hive/test2/.opq8_hive* , because the sub
> directory .opq8_hive* starts with ".", Hive will not delete it.
> hive> set hive.exec.stagingdir=./opq8;
> hive> insert overwrite table test2 select * from test3;
> Query ID = mr_20170515143940_ae48a65e-42be-4f50-b974-b713ca902867
> Total jobs = 3
> Launching Job 1 out of 3
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1494818119523_0012, Tracking URL =
> http://zdh77:8088/proxy/application_1494818119523_0012/
> Kill Command = /opt/ZDH/parcels/lib/hadoop/bin/hadoop job -kill
> job_1494818119523_0012
> Hadoop job information for Stage-1: number of mappers: 0; number of reducers: > 0
> 2017-05-15 14:40:04,547 Stage-1 map = 0%, reduce = 0%
> Ended Job = job_1494818119523_0012
> Stage-3 is selected by condition resolver.
> Stage-2 is filtered out by condition resolver.
> Stage-4 is filtered out by condition resolver.
> Moving data to directory
> hdfs://nameservice/hive/test2/.opqt8_hive_2017-05-15_14-39-40_751_1221840798987515724-1/-ext-10000
> Loading data to table default.test2
> MapReduce Jobs Launched:
> Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 SUCCESS
> Total MapReduce CPU Time Spent: 0 msec
> OK
> Time taken: 26.751 seconds
> hive>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
