[
https://issues.apache.org/jira/browse/HIVE-14688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765046#comment-15765046
]
Hive QA commented on HIVE-14688:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12844109/HIVE-14688.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 10793 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=144)
[vectorized_rcfile_columnar.q,vector_elt.q,explainuser_1.q,multi_insert.q,tez_dml.q,vector_bround.q,schema_evol_orc_acid_table.q,vector_when_case_null.q,orc_ppd_schema_evol_1b.q,vector_join30.q,vectorization_11.q,cte_3.q,update_tmp_table.q,vector_decimal_cast.q,groupby_grouping_id2.q,vector_decimal_round.q,tez_smb_empty.q,orc_merge6.q,vector_char_mapjoin1.q,vector_decimal_trailing.q,cte_5.q,tez_union.q,vector_decimal_2.q,columnStatsUpdateForStatsOptimizer_1.q,vector_outer_join3.q,schema_evol_text_vec_part_all_complex.q,tez_dynpart_hashjoin_2.q,auto_sortmerge_join_12.q,offset_limit.q,tez_union_multiinsert.q]
TestVectorizedColumnReaderBase - did not produce a TEST-*.xml file (likely
timed out) (batchId=252)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[str_to_map] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_sort_array]
(batchId=59)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_drop_partition]
(batchId=156)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_drop_table]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_based_fetch_decision]
(batchId=151)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exchange_partition_neg_incomplete_partition]
(batchId=84)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_00_unsupported_schema]
(batchId=85)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testMapWithComplexData[1]
(batchId=174)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimal
(batchId=174)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2655/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2655/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2655/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12844109 - PreCommit-HIVE-Build
> Hive drop call fails in presence of TDE
> ---------------------------------------
>
> Key: HIVE-14688
> URL: https://issues.apache.org/jira/browse/HIVE-14688
> Project: Hive
> Issue Type: Bug
> Components: Security
> Affects Versions: 1.2.1, 2.0.0
> Reporter: Deepesh Khandelwal
> Assignee: Wei Zheng
> Attachments: HIVE-14688.1.patch, HIVE-14688.2.patch,
> HIVE-14688.3.patch, HIVE-14688.4.patch
>
>
> This should be committed to when Hive moves to Hadoop 2.8
> In Hadoop 2.8.0 TDE trash collection was fixed through HDFS-8831. This
> enables us to make drop table calls for Hive managed tables where Hive
> metastore warehouse directory is in encrypted zone. However even with the
> feature in HDFS, Hive drop table currently fail:
> {noformat}
> $ hdfs crypto -listZones
> /apps/hive/warehouse key2
> $ hdfs dfs -ls /apps/hive/warehouse
> Found 1 items
> drwxrwxrwt - hdfs hdfs 0 2016-09-01 02:54
> /apps/hive/warehouse/.Trash
> hive> create table abc(a string, b int);
> OK
> Time taken: 5.538 seconds
> hive> dfs -ls /apps/hive/warehouse;
> Found 2 items
> drwxrwxrwt - hdfs hdfs 0 2016-09-01 02:54
> /apps/hive/warehouse/.Trash
> drwxrwxrwx - deepesh hdfs 0 2016-09-01 17:15
> /apps/hive/warehouse/abc
> hive> drop table if exists abc;
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to drop
> default.abc because it is in an encryption zone and trash is enabled. Use
> PURGE option to skip trash.)
> {noformat}
> The problem lies here:
> {code:title=metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java}
> private void checkTrashPurgeCombination(Path pathToData, String objectName,
> boolean ifPurge)
> ...
> if (trashEnabled) {
> try {
> HadoopShims.HdfsEncryptionShim shim =
>
> ShimLoader.getHadoopShims().createHdfsEncryptionShim(FileSystem.get(hiveConf),
> hiveConf);
> if (shim.isPathEncrypted(pathToData)) {
> throw new MetaException("Unable to drop " + objectName + "
> because it is in an encryption zone" +
> " and trash is enabled. Use PURGE option to skip trash.");
> }
> } catch (IOException ex) {
> MetaException e = new MetaException(ex.getMessage());
> e.initCause(ex);
> throw e;
> }
> }
> {code}
> As we can see that we are making an assumption that delete wouldn't be
> successful in encrypted zone. We need to modify this logic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
