Re: Hive SQL extension

2020-10-22 Thread Stamatis Zampetakis
Hi Peter, I am nowhere near being an expert but just wanted to share my thoughts. If I understand correctly you would like some syntactic sugar in Hive to support partitioning as per Iceberg. I cannot tell if that's really useful or not but from my point of view it doesn't seem a very good idea

[jira] [Created] (HIVE-24252) Improve decision model for using semijoin reducers

2020-10-09 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24252: -- Summary: Improve decision model for using semijoin reducers Key: HIVE-24252 URL: https://issues.apache.org/jira/browse/HIVE-24252 Project: Hive

[jira] [Created] (HIVE-24251) Improve bloom filter size estimation for multi column semijoin reducers

2020-10-09 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24251: -- Summary: Improve bloom filter size estimation for multi column semijoin reducers Key: HIVE-24251 URL: https://issues.apache.org/jira/browse/HIVE-24251

[jira] [Created] (HIVE-24221) Use vectorizable expression to combine multiple columns in semijoin bloom filters

2020-10-01 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24221: -- Summary: Use vectorizable expression to combine multiple columns in semijoin bloom filters Key: HIVE-24221 URL: https://issues.apache.org/jira/browse/HIVE-24221

[jira] [Created] (HIVE-24180) 'hive.txn.heartbeat.threadpool.size' is deprecated in HiveConf with no alternative

2020-09-18 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24180: -- Summary: 'hive.txn.heartbeat.threadpool.size' is deprecated in HiveConf with no alternative Key: HIVE-24180 URL: https://issues.apache.org/jira/browse/HIVE-24180

[jira] [Created] (HIVE-24179) Memory leak in HS2 DbTxnManager when compiling SHOW LOCKS statement

2020-09-18 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24179: -- Summary: Memory leak in HS2 DbTxnManager when compiling SHOW LOCKS statement Key: HIVE-24179 URL: https://issues.apache.org/jira/browse/HIVE-24179

[jira] [Created] (HIVE-24167) NPE in query 14 while generating plan for sub query predicate

2020-09-15 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24167: -- Summary: NPE in query 14 while generating plan for sub query predicate Key: HIVE-24167 URL: https://issues.apache.org/jira/browse/HIVE-24167 Project

[jira] [Created] (HIVE-24112) TestMiniLlapLocalCliDriver[dynamic_semijoin_reduction_on_aggcol] is flaky

2020-09-02 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24112: -- Summary: TestMiniLlapLocalCliDriver[dynamic_semijoin_reduction_on_aggcol] is flaky Key: HIVE-24112 URL: https://issues.apache.org/jira/browse/HIVE-24112

[jira] [Created] (HIVE-24104) NPE due to null key columns in ReduceSink after deduplication

2020-09-01 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24104: -- Summary: NPE due to null key columns in ReduceSink after deduplication Key: HIVE-24104 URL: https://issues.apache.org/jira/browse/HIVE-24104 Project

[jira] [Created] (HIVE-24031) Infinite planning time on syntactically big queries

2020-08-12 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24031: -- Summary: Infinite planning time on syntactically big queries Key: HIVE-24031 URL: https://issues.apache.org/jira/browse/HIVE-24031 Project: Hive

[jira] [Created] (HIVE-24018) Review necessity of AggregationDesc#setGenericUDAFWritableEvaluator for bloom filter aggregations

2020-08-07 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24018: -- Summary: Review necessity of AggregationDesc#setGenericUDAFWritableEvaluator for bloom filter aggregations Key: HIVE-24018 URL: https://issues.apache.org/jira/browse

[jira] [Created] (HIVE-24016) Share bloom filter construction branch in multi column semijoin reducers

2020-08-07 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-24016: -- Summary: Share bloom filter construction branch in multi column semijoin reducers Key: HIVE-24016 URL: https://issues.apache.org/jira/browse/HIVE-24016

[jira] [Created] (HIVE-23999) Unify the code creating single and multi column semijoin reducers

2020-08-06 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23999: -- Summary: Unify the code creating single and multi column semijoin reducers Key: HIVE-23999 URL: https://issues.apache.org/jira/browse/HIVE-23999 Project

[jira] [Created] (HIVE-23976) Enable vectorization for multi-col semi join reducers

2020-08-03 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23976: -- Summary: Enable vectorization for multi-col semi join reducers Key: HIVE-23976 URL: https://issues.apache.org/jira/browse/HIVE-23976 Project: Hive

Re: Hive TPC-DS metastore dumps in Postgres

2020-07-31 Thread Stamatis Zampetakis
There is now a PR [1] with various improvements over the last update. Feel free to check it out and let me know what you think. Best, Stamatis [1] https://github.com/apache/hive/pull/1347 On Mon, Jun 22, 2020 at 5:32 PM Stamatis Zampetakis wrote: > Hey guys, > > I put up a smal

[jira] [Created] (HIVE-23965) Improve plan regression tests using TPCDS30TB metastore dump and custom configs

2020-07-31 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23965: -- Summary: Improve plan regression tests using TPCDS30TB metastore dump and custom configs Key: HIVE-23965 URL: https://issues.apache.org/jira/browse/HIVE-23965

[jira] [Created] (HIVE-23964) SemanticException in query 30 while generating logical plan

2020-07-31 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23964: -- Summary: SemanticException in query 30 while generating logical plan Key: HIVE-23964 URL: https://issues.apache.org/jira/browse/HIVE-23964 Project: Hive

[jira] [Created] (HIVE-23963) UnsupportedOperationException in queries 74 and 84 while applying HiveCardinalityPreservingJoinRule

2020-07-31 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23963: -- Summary: UnsupportedOperationException in queries 74 and 84 while applying HiveCardinalityPreservingJoinRule Key: HIVE-23963 URL: https://issues.apache.org/jira

[jira] [Created] (HIVE-23946) Improve control flow and error handling in QTest dataset loading/unloading

2020-07-29 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23946: -- Summary: Improve control flow and error handling in QTest dataset loading/unloading Key: HIVE-23946 URL: https://issues.apache.org/jira/browse/HIVE-23946

[jira] [Created] (HIVE-23940) Add TPCH tables (scale factor 0.001) as qt datasets

2020-07-27 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23940: -- Summary: Add TPCH tables (scale factor 0.001) as qt datasets Key: HIVE-23940 URL: https://issues.apache.org/jira/browse/HIVE-23940 Project: Hive

[jira] [Created] (HIVE-23934) Refactor TezCompiler#markSemiJoinForDPP to avoid redundant operations in nested while

2020-07-26 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23934: -- Summary: Refactor TezCompiler#markSemiJoinForDPP to avoid redundant operations in nested while Key: HIVE-23934 URL: https://issues.apache.org/jira/browse/HIVE-23934

[jira] [Created] (HIVE-23781) Incomplete partition column stats in CachedStore may lead to wrong aggregate stats

2020-06-30 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23781: -- Summary: Incomplete partition column stats in CachedStore may lead to wrong aggregate stats Key: HIVE-23781 URL: https://issues.apache.org/jira/browse/HIVE-23781

[jira] [Created] (HIVE-23768) Metastore's update service wrongly strips partition column stats from the cache

2020-06-26 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23768: -- Summary: Metastore's update service wrongly strips partition column stats from the cache Key: HIVE-23768 URL: https://issues.apache.org/jira/browse/HIVE-23768

Re: 【Why the NULL will be filterd in HQL】

2020-06-26 Thread Stamatis Zampetakis
Hello, I think it would be easier to understand the problem if you have a query at hand: SELECT id FROM author WHERE fname != 'Victor' 0 | Victor 1 | null 2 | Alex The query should return 2 in every standard compliant SQL database. Victor != Victor evaluates to FALSE null != Victor evaluates

[jira] [Created] (HIVE-23742) Remove unintentional execution of TPC-DS query39 in qtests

2020-06-22 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23742: -- Summary: Remove unintentional execution of TPC-DS query39 in qtests Key: HIVE-23742 URL: https://issues.apache.org/jira/browse/HIVE-23742 Project: Hive

Hive TPC-DS metastore dumps in Postgres

2020-06-22 Thread Stamatis Zampetakis
Hey guys, I put up a small project on GitHub [1] with Hive metastore dumps from tpcds10tb/tpcds30tb (+partitioning) and some scripts to quickly spin up a dockerized Postgres with those loaded. Personally, I find it useful to check the plans of TPC-DS queries using the usual qtest mechanism

Re: HIVE building on ARM

2020-06-18 Thread Stamatis Zampetakis
Hello Chinna, The hudson-jobadmin privilege can be granted by PMC chairs. I don't know if there is any particular policy in Hive on who should have this privilege so I guess you should request it from Ashutosh. Best, Stamatis On Thu, Jun 18, 2020 at 12:05 PM Zoltan Haindrich wrote: > Hey

[jira] [Created] (HIVE-23684) Large underestimation in NDV stats when input and join cardinality ratio is big

2020-06-12 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23684: -- Summary: Large underestimation in NDV stats when input and join cardinality ratio is big Key: HIVE-23684 URL: https://issues.apache.org/jira/browse/HIVE-23684

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-05 Thread Stamatis Zampetakis
+1 for failing fast starting with findbugs and eventually covering the important points from checkstyle. Bes, Stamatis On Fri, Jun 5, 2020 at 9:35 AM Zoltan Haindrich wrote: > > Hey Mustafa! > > Those checks are not executed anymore in the new system. I always feeled > it a bit confusing to

Re: [DISCUSS] Disable ptest job

2020-06-05 Thread Stamatis Zampetakis
Hi Zoltan, The sooner we move away from the old system the better. It will also help to detect and solve faster any kind of problems with the new approach if there are more people using it. Also it will be cool to have junit5 :D Best, Stamatis On Fri, Jun 5, 2020 at 10:44 AM Zoltan Haindrich

Re: Open old PRs

2020-06-02 Thread Stamatis Zampetakis
Hello, I am very happy working with the new system. Many thanks Zoltan! I find the bot a good idea and I think its worth trying it out. One thing to watch out is the case where contributors are willing to push their work forward but there are no available reviewers to look to each case. I think

[jira] [Created] (HIVE-23534) NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message

2020-05-22 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23534: -- Summary: NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message Key: HIVE-23534 URL: https://issues.apache.org/jira/browse/HIVE-23534

[jira] [Created] (HIVE-23532) NPE when fetching incomplete column statistics from the metastore

2020-05-22 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23532: -- Summary: NPE when fetching incomplete column statistics from the metastore Key: HIVE-23532 URL: https://issues.apache.org/jira/browse/HIVE-23532 Project

[jira] [Created] (HIVE-23485) Bound GroupByOperator stats using largest NDV among columns

2020-05-17 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23485: -- Summary: Bound GroupByOperator stats using largest NDV among columns Key: HIVE-23485 URL: https://issues.apache.org/jira/browse/HIVE-23485 Project: Hive

[jira] [Created] (HIVE-23479) Avoid regenerating JdbcSchema for every table in a query

2020-05-15 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23479: -- Summary: Avoid regenerating JdbcSchema for every table in a query Key: HIVE-23479 URL: https://issues.apache.org/jira/browse/HIVE-23479 Project: Hive

[jira] [Created] (HIVE-23456) Upgrade Calcite version to 1.23.0

2020-05-12 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23456: -- Summary: Upgrade Calcite version to 1.23.0 Key: HIVE-23456 URL: https://issues.apache.org/jira/browse/HIVE-23456 Project: Hive Issue Type: Task

[jira] [Created] (HIVE-23453) IntelliJ compile errors in StaticPermanentFunctionChecker and TestVectorGroupByOperator

2020-05-12 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23453: -- Summary: IntelliJ compile errors in StaticPermanentFunctionChecker and TestVectorGroupByOperator Key: HIVE-23453 URL: https://issues.apache.org/jira/browse/HIVE-23453

Re: java.lang.IllegalAccessError:tried to access method com.goole.common.collect.Iterators.empty()Lcom/google/commmon/collect/UnmodifiableIterator;from class org.apache.hadoop.hive.ql.exec.FetchOperat

2020-05-09 Thread Stamatis Zampetakis
Hello, According to the release page [1] Hive 2.3.3 works with Hadoop 2.x.y (not 3.x.y) so if you want to run with Hadoop 3.2.1 try a newer version. Other than that the error looks like a classpath problem related with guava. I guess you have one Guava version coming from Hive and another