[jira] [Commented] (HIVE-13621) compute stats in certain cases fails with NPE
[ https://issues.apache.org/jira/browse/HIVE-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281785#comment-15281785 ] Pengcheng Xiong commented on HIVE-13621: reran all those Spark tests, none of them fail. Also check the other failures and they are unrelated. Pushed to master. Thanks [~vikram.dixit] and [~hagleitn] for the review and comments! > compute stats in certain cases fails with NPE > - > > Key: HIVE-13621 > URL: https://issues.apache.org/jira/browse/HIVE-13621 > Project: Hive > Issue Type: Bug > Components: HBase Metastore, Metastore >Affects Versions: 2.1.0, 2.0.1 >Reporter: Vikram Dixit K >Assignee: Pengcheng Xiong > Attachments: HIVE-13621.1.patch, HIVE-13621.2.patch > > > {code} > FAILED: NullPointerException null > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatistics(StatsUtils.java:693) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.convertColStats(StatsUtils.java:739) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:728) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:183) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124){code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13621) compute stats in certain cases fails with NPE
[ https://issues.apache.org/jira/browse/HIVE-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281366#comment-15281366 ] Hive QA commented on HIVE-13621: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12803154/HIVE-13621.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 66 failed/errored test(s), 9215 tests executed *Failed tests:* {noformat} TestCompactor - did not produce a TEST-*.xml file TestHWISessionManager - did not produce a TEST-*.xml file TestMiniLlapCliDriver - did not produce a TEST-*.xml file TestMiniTezCliDriver-explainuser_4.q-update_after_multiple_inserts.q-mapreduce2.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-groupby2.q-tez_dynpart_hashjoin_1.q-custom_input_output_format.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-join1.q-mapjoin_decimal.q-union5.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-smb_cache.q-transform_ppr2.q-vector_outer_join0.q-and-5-more - did not produce a TEST-*.xml file TestNegativeCliDriver-udf_invalid.q-nopart_insert.q-insert_into_with_schema.q-and-734-more - did not produce a TEST-*.xml file TestSparkCliDriver-skewjoinopt15.q-join39.q-avro_joins_native.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join18_multi_distinct org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join2 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join4 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_5 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_avro_decimal_native org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin12 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin4 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_column_access_stats org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby1_map_skew org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby5_map org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby6_map org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_position org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join0 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join13 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join18_multi_distinct org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join30 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join_array org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join_cond_pushdown_2 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join_cond_pushdown_unqual4 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join_reorder3 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ptf_matchpath org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_skewjoinopt12 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_skewjoinopt5 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats16 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_partscan_1_23 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_udf_percentile org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union14 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union24 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union31 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union34 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vector_groupby_3 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_0 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_2 org.apache.hadoop.hive.llap.daemon.impl.comparator.TestFirstInFirstOutComparator.testWaitQueueComparatorWithinDagPriority org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation org.apache.hadoop.hive.metastore.TestFilterHooks.org.apache.hadoop.hive.metastore.TestFilterHooks org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf.org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf org.apache.hadoop.hive.metastore.TestHiveMetaStoreWithEnvironmentContext.testEnvironmentContext org.apache.hadoop.hive.metastore.TestMetaStoreInitListener.testMetaStoreInitListener org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.org.apache.hadoop.hive.metastore.TestMetaStoreMetrics
[jira] [Commented] (HIVE-13621) compute stats in certain cases fails with NPE
[ https://issues.apache.org/jira/browse/HIVE-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278431#comment-15278431 ] Vikram Dixit K commented on HIVE-13621: --- +1 pending tests. > compute stats in certain cases fails with NPE > - > > Key: HIVE-13621 > URL: https://issues.apache.org/jira/browse/HIVE-13621 > Project: Hive > Issue Type: Bug > Components: HBase Metastore, Metastore >Affects Versions: 2.1.0, 2.0.1 >Reporter: Vikram Dixit K >Assignee: Pengcheng Xiong > Attachments: HIVE-13621.1.patch, HIVE-13621.2.patch > > > {code} > FAILED: NullPointerException null > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatistics(StatsUtils.java:693) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.convertColStats(StatsUtils.java:739) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:728) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:183) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124){code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13621) compute stats in certain cases fails with NPE
[ https://issues.apache.org/jira/browse/HIVE-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264025#comment-15264025 ] Hive QA commented on HIVE-13621: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12800922/HIVE-13621.1.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 10018 tests executed *Failed tests:* {noformat} TestHWISessionManager - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_deleteAnalyze org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nomore_ambiguous_table_col org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_regexp_extract org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_clustern3 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_clustern4 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_nonkey_groupby org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_selectDistinctStarNeg_2 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_subquery_shared_alias org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udtf_not_supported1 org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote.org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote org.apache.hadoop.hive.metastore.TestFilterHooks.org.apache.hadoop.hive.metastore.TestFilterHooks org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf.testGetMetaConfDefault org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadTableFailure org.apache.hive.hcatalog.listener.TestDbNotificationListener.dropDatabase {noformat} Test results: http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/118/testReport Console output: http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/118/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-118/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 17 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12800922 - PreCommit-HIVE-MASTER-Build > compute stats in certain cases fails with NPE > - > > Key: HIVE-13621 > URL: https://issues.apache.org/jira/browse/HIVE-13621 > Project: Hive > Issue Type: Bug > Components: HBase Metastore, Metastore >Affects Versions: 2.1.0, 2.0.1 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-13621.1.patch > > > {code} > FAILED: NullPointerException null > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatistics(StatsUtils.java:693) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.convertColStats(StatsUtils.java:739) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:728) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:183) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124){code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13621) compute stats in certain cases fails with NPE
[ https://issues.apache.org/jira/browse/HIVE-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263052#comment-15263052 ] Gunther Hagleitner commented on HIVE-13621: --- Should be able to use: {noformat} com.google.protobuf.ByteString.EMPTY {noformat} instead of your 1 byte array. > compute stats in certain cases fails with NPE > - > > Key: HIVE-13621 > URL: https://issues.apache.org/jira/browse/HIVE-13621 > Project: Hive > Issue Type: Bug > Components: HBase Metastore, Metastore >Affects Versions: 2.1.0, 2.0.1 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-13621.1.patch > > > {code} > FAILED: NullPointerException null > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatistics(StatsUtils.java:693) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.convertColStats(StatsUtils.java:739) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:728) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:183) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124){code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13621) compute stats in certain cases fails with NPE
[ https://issues.apache.org/jira/browse/HIVE-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263036#comment-15263036 ] Gunther Hagleitner commented on HIVE-13621: --- The explain plan in the test shows this: {noformat} Select Operator [SEL_9] (rows=9223372036854775807 width=1) {noformat} This seems horribly wrong, no? (might be unrelated, but looks serious). > compute stats in certain cases fails with NPE > - > > Key: HIVE-13621 > URL: https://issues.apache.org/jira/browse/HIVE-13621 > Project: Hive > Issue Type: Bug > Components: HBase Metastore, Metastore >Affects Versions: 2.1.0, 2.0.1 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-13621.1.patch > > > {code} > FAILED: NullPointerException null > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatistics(StatsUtils.java:693) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.convertColStats(StatsUtils.java:739) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:728) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:183) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136) > at > org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124){code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)