[
https://issues.apache.org/jira/browse/HIVE-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297482#comment-15297482
]
Hive QA commented on HIVE-11233:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12805513/HIVE-11233.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 50 failed/errored test(s), 10071 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_join30.q-vector_decimal_10_0.q-acid_globallimit.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vectorized_parquet.q-insert_values_non_partitioned.q-schema_evol_orc_nonvec_mapwork_part.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-parallel_join1.q-escape_distributeby1.q-auto_sortmerge_join_7.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-table_access_keys_stats.q-bucketsortoptimize_insert_4.q-runtime_skewjoin_mapjoin_spark.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-union29.q-join23.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_external_table_ppd
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_storage_queries
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join21
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cross_product_check_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_llapdecider
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_mapwork_part_all_primitive
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_script_env_var1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_fsstat
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union_with_udf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union4
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_interval_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_null_projection
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_12
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_4
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_ptf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_shufflejoin
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testPreemptionQueueComparator
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hadoop.hive.metastore.TestFilterHooks.org.apache.hadoop.hive.metastore.TestFilterHooks
org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf.org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf
org.apache.hadoop.hive.metastore.TestMetaStoreEventListenerOnlyOnCommit.testEventStatus
org.apache.hadoop.hive.metastore.TestMetaStoreInitListener.testMetaStoreInitListener
org.apache.hadoop.hive.metastore.TestRemoteUGIHiveMetaStoreIpAddress.testIpAddress
org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testLockTimeout
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropPartition
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadDbFailure
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadDbSuccess
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadTableFailure
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadTableSuccess
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.testPigPopulation
org.apache.hive.jdbc.TestSSL.testSSLFetchHttp
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.org.apache.hive.service.TestHS2ImpersonationWithRemoteMS
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/369/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/369/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-369/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 50 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12805513 - PreCommit-HIVE-MASTER-Build
> Include Apache Phoenix support in HBaseStorageHandler
> -----------------------------------------------------
>
> Key: HIVE-11233
> URL: https://issues.apache.org/jira/browse/HIVE-11233
> Project: Hive
> Issue Type: New Feature
> Components: HBase Handler
> Affects Versions: 1.2.1, 2.0.0
> Reporter: Svetozar Ivanov
> Assignee: Svetozar Ivanov
> Labels: Binary, Hbase, Numeric, Phoenix, Sortable
> Attachments: HIVE-11233-branch-1.2.patch,
> HIVE-11233-branch-2.0.patch, HIVE-11233.1.patch, HIVE-11233.2.patch,
> HIVE-11233.3.patch, HIVE-11233.patch
>
>
> Currently HBaseStorageHandler doesn't provide mechanism for storage of binary
> sortable key and values. It is necessary when given HBase table is used for
> persistence by Apache Hive and Apache Phoenix. In that way all byte arrays
> read or written by Hive will be compatible with binary sortable format used
> in Phoenix.
> It turns out the major difference is in all numeric data types accordingly
> officially provided documentation -
> https://phoenix.apache.org/language/datatypes.html.
> That's how I'm using it in my code:
> {code}
> private static String buildWithSerDeProperties(TableDescriptor
> tableDescriptor) {
> Map<String, String> serdePropertiesMap = new HashMap<>();
> serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_NAME,
> tableDescriptor.getTableName());
> serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_DEFAULT_STORAGE_TYPE,
> BINARY_STORAGE_TYPE);
> serdePropertiesMap.put(HBaseSerDe.HBASE_COLUMNS_MAPPING,
> buildHBaseColumnsDefinition(tableDescriptor));
> serdePropertiesMap.put(HBaseSerDe.HBASE_VALUE_FACTORY_CLASS,
> PhoenixValueFactory.class.getName());
> /* Use different key factory for simple and composite primary key */
> if (tableDescriptor.getPkDescriptors().size() == 1) {
> serdePropertiesMap.put(HBaseSerDe.HBASE_KEY_FACTORY_CLASS,
> PhoenixKeyFactory.class.getName());
> } else {
> serdePropertiesMap.put(HBaseSerDe.HBASE_COMPOSITE_KEY_FACTORY,
> PhoenixCompositeKeyFactory.class.getName());
> }
> String serDeProperties = serdePropertiesMap.entrySet().stream()
> .map(e -> quoteInSingleQuotes(e.getKey()) + " = " +
> quoteInSingleQuotes(e.getValue()))
> .collect(Collectors.joining(COLUMNS_SEPARATOR));
> logger.debug("SERDEPROPERTIES are [{}]", serDeProperties);
> return serDeProperties;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
