Re: Review Request 69257: HIVE-20842 Fix logic introduced in HIVE-20660 to estimate statistics for group by
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69257/ --- (Updated Nov. 13, 2018, 12:51 a.m.) Review request for hive and Ashutosh Chauhan. Bugs: HIVE-20842 https://issues.apache.org/jira/browse/HIVE-20842 Repository: hive-git Description --- See jIRA Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java f0b41f36f3 ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java 32fba6c8ff ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets2.q.out 7bee405977 ql/src/test/results/clientpositive/llap/vectorization_short_regress.q.out f929706757 ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out 32b0e3ec2a ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out 184d281df0 ql/src/test/results/clientpositive/perf/tez/constraints/query77.q.out 78f7c28b7a ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out dbaecf8fe8 ql/src/test/results/clientpositive/perf/tez/constraints/query95.q.out fd709f99dd ql/src/test/results/clientpositive/perf/tez/query5.q.out 2ce689b1bb ql/src/test/results/clientpositive/perf/tez/query70.q.out 4650ef66fc ql/src/test/results/clientpositive/perf/tez/query80.q.out 223e61aa24 Diff: https://reviews.apache.org/r/69257/diff/7/ Changes: https://reviews.apache.org/r/69257/diff/6-7/ Testing --- Thanks, Vineet Garg
Re: Review Request 69257: HIVE-20842 Fix logic introduced in HIVE-20660 to estimate statistics for group by
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69257/ --- (Updated Nov. 12, 2018, 10:46 p.m.) Review request for hive and Ashutosh Chauhan. Changes --- Rebased patch Bugs: HIVE-20842 https://issues.apache.org/jira/browse/HIVE-20842 Repository: hive-git Description --- See jIRA Diffs (updated) - llap-server/src/main/resources/package.py c48ff79691 ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java f0b41f36f3 ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java 32fba6c8ff ql/src/test/results/clientpositive/annotate_stats_groupby.q.out fe30d3197c ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out 91369679aa ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 4e6885d9a2 ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out 5de5a69f87 ql/src/test/results/clientpositive/llap/check_constraint.q.out 2b1564879c ql/src/test/results/clientpositive/llap/constraints_optimization.q.out bf60646636 ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out efa2dd818a ql/src/test/results/clientpositive/llap/correlationoptimizer6.q.out 93a3017696 ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out bd3c7769a4 ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out 30074abaf2 ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out dca0ebdab8 ql/src/test/results/clientpositive/llap/except_distinct.q.out c155a73c96 ql/src/test/results/clientpositive/llap/explainuser_1.q.out c86450aae2 ql/src/test/results/clientpositive/llap/explainuser_2.q.out 1dd9a0ff30 ql/src/test/results/clientpositive/llap/explainuser_4.q.out 8de83793ac ql/src/test/results/clientpositive/llap/intersect_all.q.out dbb77d1abc ql/src/test/results/clientpositive/llap/intersect_distinct.q.out 604c7bbd63 ql/src/test/results/clientpositive/llap/intersect_merge.q.out b19fd2c4ec ql/src/test/results/clientpositive/llap/limit_pushdown.q.out a84a7b3db3 ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 8d5848bcd3 ql/src/test/results/clientpositive/llap/mrr.q.out a8aceea293 ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 133d8af9e9 ql/src/test/results/clientpositive/llap/parallel.q.out 692bb8ca74 ql/src/test/results/clientpositive/llap/parallel_colstats.q.out 91a450accf ql/src/test/results/clientpositive/llap/ptf.q.out b719f73566 ql/src/test/results/clientpositive/llap/reduce_deduplicate_distinct.q.out 8d04800040 ql/src/test/results/clientpositive/llap/reduce_deduplicate_extended.q.out 54dc0f7b8f ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out 17601acc2d ql/src/test/results/clientpositive/llap/sharedworkext.q.out e3488151bf ql/src/test/results/clientpositive/llap/sqlmerge_stats.q.out cd178cff4c ql/src/test/results/clientpositive/llap/subquery_in.q.out 9cfc960bf0 ql/src/test/results/clientpositive/llap/subquery_in_having.q.out 3839696882 ql/src/test/results/clientpositive/llap/subquery_multi.q.out 7b00d69754 ql/src/test/results/clientpositive/llap/subquery_notin.q.out 29d8bbfb48 ql/src/test/results/clientpositive/llap/subquery_scalar.q.out c43ad9188b ql/src/test/results/clientpositive/llap/subquery_select.q.out 0435530467 ql/src/test/results/clientpositive/llap/subquery_views.q.out aa50b468c3 ql/src/test/results/clientpositive/llap/tez_join_hash.q.out 2ac8400576 ql/src/test/results/clientpositive/llap/tez_union2.q.out ef0d4bd71a ql/src/test/results/clientpositive/llap/tez_union_multiinsert.q.out 05d259b0d9 ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out 93791ac8e0 ql/src/test/results/clientpositive/llap/unionDistinct_3.q.out 5337820ab0 ql/src/test/results/clientpositive/llap/vector_adaptor_usage_mode.q.out 52b17cf36b ql/src/test/results/clientpositive/llap/vector_char_2.q.out 1ba0ab6920 ql/src/test/results/clientpositive/llap/vector_distinct_2.q.out e72e398e4b ql/src/test/results/clientpositive/llap/vector_groupby_3.q.out 3ea544e4b8 ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets2.q.out 7bee405977 ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out 3696cad941 ql/src/test/results/clientpositive/llap/vector_outer_reference_windowed.q.out fcde000739 ql/src/test/results/clientpositive/llap/vector_windowing.q.out 8e8c445af7 ql/src/test/results/clientpositive/llap/vector_windowing_gby2.q.out 5943548a6c ql/src/test/results/clientpositive/llap/vectorization_limit.q.out 3dc640a300 ql/src/test/results/clientpositive/llap/vectorization_short_regress.q.out f929706757 ql/src/test/results/clientpositive/llap/vectorized_ptf.q.out 56e81aa819 ql/src/test
[jira] [Created] (HIVE-20908) Avoid multiple getTableMeta calls per database
Rajkumar Singh created HIVE-20908: - Summary: Avoid multiple getTableMeta calls per database Key: HIVE-20908 URL: https://issues.apache.org/jira/browse/HIVE-20908 Project: Hive Issue Type: Bug Reporter: Rajkumar Singh following HIVE-19432, we are doing getTableMeta for each authorized db instead of that we can pass pattern for metastore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20907) TestGetPartitionsUsingProjectionAndFilterSpecs is flaky
Antal Sinkovits created HIVE-20907: -- Summary: TestGetPartitionsUsingProjectionAndFilterSpecs is flaky Key: HIVE-20907 URL: https://issues.apache.org/jira/browse/HIVE-20907 Project: Hive Issue Type: Bug Affects Versions: 4.0.0 Reporter: Antal Sinkovits private void verifyLocations(List origPartitions, StorageDescriptor sharedSD, List partitionWithoutSDS) method expects, that the order of the two list are the same. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20906) Are Hive locks really ephemeral ?
Rajesh KSV created HIVE-20906: - Summary: Are Hive locks really ephemeral ? Key: HIVE-20906 URL: https://issues.apache.org/jira/browse/HIVE-20906 Project: Hive Issue Type: Bug Reporter: Rajesh KSV I am using Hive 2.3.3. According to docs ([https://cwiki.apache.org/confluence/display/Hive/Locking]), they are ephemeral. But in the Zookeeper and Hive source Code, they seemed to be non-ephemeral. I ran the following query in Hive. *lock table discovery.defaultns* When I see the lock in zookeeper, it doesn't look like ephemeral node. [zk: localhost:2181(CONNECTED) 55] get /hive_zookeeper_namespace/discovery/defaultns/LOCK-EXCLUSIVE-00 hive_20181112191555_d6640955-2806-4fa7-9e52-30d04c377a04:1542030355565:EXPLICIT:lock table discovery.defaultns exclusive:10.33.222.139 cZxid = 0x2400296808 ctime = Mon Nov 12 19:15:55 IST 2018 mZxid = 0x2400296808 mtime = Mon Nov 12 19:15:55 IST 2018 pZxid = 0x2400296808 cversion = 0 dataVersion = 0 aclVersion = 0 *ephemeralOwner = 0x0* dataLength = 134 numChildren = 0 Also in the code - [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveTxnManagerImpl.java], the lock is KeepAlive/PERSISTENT and not EPHIMERAL . *HiveLock lck = lockMgr.lock(new HiveLockObject(tbl, lockData), mode, true);* Is it a bug or am I missing something? Thanks! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20905) querying streaming table fails with out of memory exception
mahesh kumar behera created HIVE-20905: -- Summary: querying streaming table fails with out of memory exception Key: HIVE-20905 URL: https://issues.apache.org/jira/browse/HIVE-20905 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 4.0.0 Reporter: mahesh kumar behera Assignee: mahesh kumar behera Fix For: 4.0.0 Streaming app was ran for 24hrs post which it went down due authentication issue . The table was accessible for 12hrs into the run, however currently querying the table fails with OOM exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20904) Yetus fails to resolve module dependencies due to usage of exec plugin in metastore-server
Antal Sinkovits created HIVE-20904: -- Summary: Yetus fails to resolve module dependencies due to usage of exec plugin in metastore-server Key: HIVE-20904 URL: https://issues.apache.org/jira/browse/HIVE-20904 Project: Hive Issue Type: Bug Reporter: Antal Sinkovits Assignee: Antal Sinkovits metastore-server uses exec-maven-plugin to generate metastore-site.xml.template with ConfTemplatePrinter. It expects some arguments. Because yetus also uses the exec-maven-plugin to determine the order of the modules to be built, but with zero params, the execution fails. https://github.com/apache/yetus/blob/6ebaa1119e611db14f219e289e33ab8ac5c254a7/precommit/src/main/shell/test-patch.d/maven.sh#L658 Steps to reproduce the issue: mvn -q exec:exec -Dexec.executable=pwd -Dexec.args='' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
