[jira] [Created] (HIVE-24013) Move anti join conversion after join reordering rule

2020-08-06 Thread mahesh kumar behera (Jira)
mahesh kumar behera created HIVE-24013:
--

 Summary: Move anti join conversion after join reordering rule
 Key: HIVE-24013
 URL: https://issues.apache.org/jira/browse/HIVE-24013
 Project: Hive
  Issue Type: Bug
Reporter: mahesh kumar behera
Assignee: mahesh kumar behera


The current anti join conversion does not check for null filters on right side 
of join if it's within OR conditions. Only those filters separated by AND 
conditions are supported. For example queries like "select t1.fld from tbl1 t1 
left join tbl2 t2 on t1.fld = t2.fld where t2.fld is null or t2.fld1 is null" 
are not converted to anti join. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24012) Support for rewriting with materialized views containing grouping sets

2020-08-06 Thread Jesus Camacho Rodriguez (Jira)
Jesus Camacho Rodriguez created HIVE-24012:
--

 Summary: Support for rewriting with materialized views containing 
grouping sets
 Key: HIVE-24012
 URL: https://issues.apache.org/jira/browse/HIVE-24012
 Project: Hive
  Issue Type: Sub-task
  Components: Materialized views
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


Rewriting is not triggered for materialized views containing grouping sets. 
This issue implements an extension from Hive side to trigger additional 
rewritings for materialized views containing grouping sets.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Triggering tests for Hive PR or patch

2020-08-06 Thread Chao Sun
Hi,

Does anyone know if github PR works for branches other than master? and if
so what is the way to trigger tests? if not, does attaching patch ending
with branch-2.3.patch still work?

Thanks,
Chao


[jira] [Created] (HIVE-24011) Flaky test AsyncResponseHandlerTest

2020-08-06 Thread Pravin Sinha (Jira)
Pravin Sinha created HIVE-24011:
---

 Summary: Flaky test AsyncResponseHandlerTest
 Key: HIVE-24011
 URL: https://issues.apache.org/jira/browse/HIVE-24011
 Project: Hive
  Issue Type: Task
Reporter: Pravin Sinha


[http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-1352/2/tests/]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24010) Support physical transformations for EXECUTE statement

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24010:
--

 Summary: Support physical transformations for EXECUTE statement
 Key: HIVE-24010
 URL: https://issues.apache.org/jira/browse/HIVE-24010
 Project: Hive
  Issue Type: Sub-task
  Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg


Currently these physical transformations are run during PREPARE statement only 
which could miss transforming due to lack of stats or due to lack of actual 
literal/constants.
Ideally once the params are bounded these transformations should be run again.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24009) Support partition pruning for EXECUTE statement

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24009:
--

 Summary: Support partition pruning for EXECUTE statement
 Key: HIVE-24009
 URL: https://issues.apache.org/jira/browse/HIVE-24009
 Project: Hive
  Issue Type: Sub-task
Reporter: Vineet Garg
Assignee: Vineet Garg


Current partition pruning (compile time) isn't kicked in for EXECUTE statements.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24008) Improve Explain plan output for EXECUTE queries

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24008:
--

 Summary: Improve Explain plan output for EXECUTE queries
 Key: HIVE-24008
 URL: https://issues.apache.org/jira/browse/HIVE-24008
 Project: Hive
  Issue Type: Sub-task
Reporter: Vineet Garg
Assignee: Vineet Garg


Currently explain on EXECUTE shows prepare query text instead of execute query.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24007) Support for EXECUTE/PREPARE operation authorization

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24007:
--

 Summary: Support for EXECUTE/PREPARE operation authorization
 Key: HIVE-24007
 URL: https://issues.apache.org/jira/browse/HIVE-24007
 Project: Hive
  Issue Type: Sub-task
  Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg


Currently both of these operations are created with null input and output 
privileges. This needs to be investigated and fixed/improved to support proper 
authorization.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24006) Support dynamic expressions/params in SELECT clause

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24006:
--

 Summary: Support dynamic expressions/params in SELECT clause
 Key: HIVE-24006
 URL: https://issues.apache.org/jira/browse/HIVE-24006
 Project: Hive
  Issue Type: Sub-task
  Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg


Currently it is only supported in WHERE/HAVING clause.

 

One of the challenge here is type inference.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24005) Reduce memory footprint of cached plan

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24005:
--

 Summary: Reduce memory footprint of cached plan
 Key: HIVE-24005
 URL: https://issues.apache.org/jira/browse/HIVE-24005
 Project: Hive
  Issue Type: Sub-task
  Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg


Currently whole {{SemanticAnalyzer}} object is cached. It is not necessary to 
cache the whole object (caching requires set of tasks, query config and some 
other info). 

May be create a CachedPlan object.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24004) Improve performance for filter hook for superuser path

2020-08-06 Thread Sam An (Jira)
Sam An created HIVE-24004:
-

 Summary: Improve performance for filter hook for superuser path
 Key: HIVE-24004
 URL: https://issues.apache.org/jira/browse/HIVE-24004
 Project: Hive
  Issue Type: Improvement
  Components: Hive
Affects Versions: 4.0.0
Reporter: Sam An
Assignee: Sam An


In HiveMetastoreAuthoriver, the sequence of creating the authorizer can be 
optimized so that for super user when we can skip authorization, we can skip 
creating authorizer. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24003) Stats annotation/collection for EXECUTE statements

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24003:
--

 Summary: Stats annotation/collection for EXECUTE statements
 Key: HIVE-24003
 URL: https://issues.apache.org/jira/browse/HIVE-24003
 Project: Hive
  Issue Type: Sub-task
  Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg


Once the parameters are bounded we should re-run stats annotation so that all 
the physical statistics based optimizations could be run.

Currently stats are either collected (and skipped for dynamic param 
expressions) during PREPARE.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24002) Allow 'expressions' instead of 'constant literal' for EXECUTE

2020-08-06 Thread Vineet Garg (Jira)
Vineet Garg created HIVE-24002:
--

 Summary: Allow 'expressions' instead of 'constant literal' for 
EXECUTE
 Key: HIVE-24002
 URL: https://issues.apache.org/jira/browse/HIVE-24002
 Project: Hive
  Issue Type: Sub-task
  Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg


Currently EXECUTE statement grammar allows constant literal only. This prevents 
negative literals or constant expressions like {{1+2}}. Changing it to 
{{expression}} type will allow broader type of literals. Note that this will 
require additional error checking after parsing to only allow "acceptable" 
literals.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24001) Don't cache MapWork in tez/ObjectCache during query-based compaction

2020-08-06 Thread Karen Coppage (Jira)
Karen Coppage created HIVE-24001:


 Summary: Don't cache MapWork in tez/ObjectCache during query-based 
compaction
 Key: HIVE-24001
 URL: https://issues.apache.org/jira/browse/HIVE-24001
 Project: Hive
  Issue Type: Bug
Reporter: Karen Coppage
Assignee: Karen Coppage


Query-based major compaction can fail intermittently with the following issue:

{code:java}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: One writer is 
supposed to handle only one bucket. We saw these 2 different buckets: 1 and 6
  at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDFValidateAcidSortOrder.evaluate(GenericUDFValidateAcidSortOrder.java:77)
{code}

This is consistently preceded in the application log with:

{code:java}
 [INFO] [TezChild] |tez.ObjectCache|: Found 
hive_20200804185133_f04cca69-fa30-4f1b-a5fe-80fc2d749f48_Map 1__MAP_PLAN__ in 
cache with value: org.apache.hadoop.hive.ql.plan.MapWork@74652101
{code}

Alternatively, when MapRecordProcessor doesn't find mapWork in tez/ObjectCache 
(but instead caches mapWork), major compaction succeeds.

The failure happens because, if MapWork is reused, 
GenericUDFValidateAcidSortOrder (which is called during compaction) is also 
reused on splits belonging to two different buckets, which produces an error.

Solution is to avoid storing MapWork in the ObjectCache during query-based 
compaction.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-24000) Put exclusive MERGE INSERT under the feature flag

2020-08-06 Thread Denys Kuzmenko (Jira)
Denys Kuzmenko created HIVE-24000:
-

 Summary: Put exclusive MERGE INSERT under the feature flag
 Key: HIVE-24000
 URL: https://issues.apache.org/jira/browse/HIVE-24000
 Project: Hive
  Issue Type: Bug
Reporter: Denys Kuzmenko






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-23999) Unify the code creating single and multi column semijoin reducers

2020-08-06 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created HIVE-23999:
--

 Summary: Unify the code creating single and multi column semijoin 
reducers
 Key: HIVE-23999
 URL: https://issues.apache.org/jira/browse/HIVE-23999
 Project: Hive
  Issue Type: Improvement
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis


In HIVE-21196, we add a transformation capable of merging single column 
semijoin reducers to multi column semijoin reducer.

The code for creating multi-column semijoin reducers in SemiJoinReductionMerge 
presents some similarities with the code creating single-column semijoin 
reducers in DynamicPartitionPruningOptimization.

Possibly we could refactor the respective parts to unify the creation logic of 
semijoin reducers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)