date:20191128

Re: Review Request 71821: HIVE-22544

2019-11-28 Thread Krisztian Kasa


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71821/
---

(Updated Nov. 29, 2019, 7:06 a.m.)


Review request for hive and Jesús Camacho Rodríguez.


Bugs: HIVE-22544
https://issues.apache.org/jira/browse/HIVE-22544


Repository: hive-git


Description
---

Disable null sort order at user level


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 77c8300e32 
  ql/src/java/org/apache/hadoop/hive/ql/plan/TopNKeyDesc.java dc45800d74 
  ql/src/test/results/clientpositive/llap/constprog_dpp.q.out 11af918604 
  ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out 90dd54b78e 
  ql/src/test/results/clientpositive/llap/cte_mat_3.q.out 2433e99dbb 
  ql/src/test/results/clientpositive/llap/cte_mat_4.q.out 52ed4a78b3 
  ql/src/test/results/clientpositive/llap/cte_mat_5.q.out fac092947a 
  ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out a095fcc0ae 
  ql/src/test/results/clientpositive/llap/dynamic_semijoin_user_level.q.out 
4b9035e5b9 
  ql/src/test/results/clientpositive/llap/empty_join.q.out b19ce14c78 
  ql/src/test/results/clientpositive/llap/estimate_pkfk_filtered_fk.q.out 
b74a20f305 
  ql/src/test/results/clientpositive/llap/estimate_pkfk_nocond.q.out 5a58b13b8c 
  ql/src/test/results/clientpositive/llap/estimate_pkfk_push.q.out 2af0fb5ee3 
  ql/src/test/results/clientpositive/llap/explainanalyze_2.q.out f5b2e4b294 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 735296f814 
  ql/src/test/results/clientpositive/llap/explainuser_2.q.out b90fb55fd1 
  ql/src/test/results/clientpositive/llap/explainuser_4.q.out 3504d17875 
  ql/src/test/results/clientpositive/llap/groupby_groupingset_bug.q.out 
a8c1d56ef7 
  ql/src/test/results/clientpositive/llap/reopt_dpp.q.out 141b2b617d 
  ql/src/test/results/clientpositive/llap/retry_failure_reorder.q.out 
9e1c249ab7 
  ql/src/test/results/clientpositive/llap/retry_failure_stat_changes.q.out 
204fa8f711 
  ql/src/test/results/clientpositive/llap/runtime_stats_hs2.q.out f88aa718f8 
  ql/src/test/results/clientpositive/llap/runtime_stats_merge.q.out a11e81f1bc 
  ql/src/test/results/clientpositive/llap/temp_table_llap_partitioned.q.out 
bd61928273 
  ql/src/test/results/clientpositive/llap/windowing_gby.q.out 0888fd979e 
  ql/src/test/results/clientpositive/perf/tez/constraints/mv_query44.q.out 
6a67ca6823 
  ql/src/test/results/clientpositive/perf/tez/constraints/query1.q.out 
211e92ecf4 
  ql/src/test/results/clientpositive/perf/tez/constraints/query10.q.out 
7ad55b5c5f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query11.q.out 
2deb5827a5 
  ql/src/test/results/clientpositive/perf/tez/constraints/query12.q.out 
035c908ec3 
  ql/src/test/results/clientpositive/perf/tez/constraints/query13.q.out 
b27552f901 
  ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out 
9b07ceb3a7 
  ql/src/test/results/clientpositive/perf/tez/constraints/query15.q.out 
7c0e6cf8cd 
  ql/src/test/results/clientpositive/perf/tez/constraints/query16.q.out 
377307f6e3 
  ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out 
44fd104a9b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query18.q.out 
b65bebc22b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query19.q.out 
d313b21add 
  ql/src/test/results/clientpositive/perf/tez/constraints/query2.q.out 
e1308a315a 
  ql/src/test/results/clientpositive/perf/tez/constraints/query20.q.out 
a10979641a 
  ql/src/test/results/clientpositive/perf/tez/constraints/query21.q.out 
d944adec04 
  ql/src/test/results/clientpositive/perf/tez/constraints/query22.q.out 
0e1f13e65e 
  ql/src/test/results/clientpositive/perf/tez/constraints/query23.q.out 
761369c88b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query24.q.out 
8b4fea7d80 
  ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out 
1ca55ca290 
  ql/src/test/results/clientpositive/perf/tez/constraints/query26.q.out 
90a56acafc 
  ql/src/test/results/clientpositive/perf/tez/constraints/query27.q.out 
58b2b86cdf 
  ql/src/test/results/clientpositive/perf/tez/constraints/query28.q.out 
9634957a52 
  ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out 
073ae52f1f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query3.q.out 
50beff8772 
  ql/src/test/results/clientpositive/perf/tez/constraints/query30.q.out 
6aa98f2bdd 
  ql/src/test/results/clientpositive/perf/tez/constraints/query31.q.out 
3f0e054d0c 
  ql/src/test/results/clientpositive/perf/tez/constraints/query32.q.out 
f3f381a26e 
  ql/src/test/results/clientpositive/perf/tez/constraints/query33.q.out 
fbab5700b7 
  ql/src/test/results/clientpositive/perf/tez/constraints/query34.q.out 
8f730c8f3a 
  ql/src/test/results/clientpositive/perf/tez/constraints/query35.q.out 
4ca9d3cb61

[jira] [Created] (HIVE-22563) Required field 'client_protocol' is unset (hive server backward compatibility)

2019-11-28 Thread ZhangZhiCheng (Jira)

ZhangZhiCheng created HIVE-22563:


 Summary: Required field 'client_protocol' is unset (hive server 
backward compatibility)
 Key: HIVE-22563
 URL: https://issues.apache.org/jira/browse/HIVE-22563
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 2.3.6, 1.2.1
Reporter: ZhangZhiCheng
 Attachments: image-2019-11-29-10-20-46-982.png

I try to connect to hive server 1.2.1 by using hive client 
(hive-jdbc-2.3.6.jar) , then I got this issue "Required field 'client_protocol' 
is unset".  is that means hive server 1.2.1 has no backward compatibility for 
newer hive client version?   

!image-2019-11-29-10-20-46-982.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-22562) Harmonize SessionState.getUserName

2019-11-28 Thread Zoltan Haindrich (Jira)

Zoltan Haindrich created HIVE-22562:
---

 Summary: Harmonize SessionState.getUserName
 Key: HIVE-22562
 URL: https://issues.apache.org/jira/browse/HIVE-22562
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


we might have 2 different user names at the same time:

* 
[getUserName()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L1912]
** a method which relies on the userName field of the SessionState
* 
[getUserFromAuthenticator()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L1291]
** a method which uses the authenticator to do the heavy lifting
* there all kind of interesting call sites like:
** there are some which are [prefering the authenticator over 
getUserName()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L254]
** there are some which [use getUserName() regardless authenticator, but have 
fixme|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/Driver.java#L1669]
** and there are some which are just using the authenticator with or without 
notes/etc





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-22561) Data loss on map join for bucketed, partitioned table

2019-11-28 Thread Aditya Shah (Jira)

Aditya Shah created HIVE-22561:
--

 Summary: Data loss on map join for bucketed, partitioned table
 Key: HIVE-22561
 URL: https://issues.apache.org/jira/browse/HIVE-22561
 Project: Hive
  Issue Type: Bug
Affects Versions: 3.1.2
Reporter: Aditya Shah
 Attachments: Screenshot 2019-11-28 at 8.45.17 PM.png, 
image-2019-11-28-20-46-25-432.png

A map join on a column (which is neither involved in bucketing and partition) 
causes data loss. 

Steps to reproduce:

Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.

Create tables:

 
{code:java}
CREATE TABLE `testj2`(
  `id` int, 
  `bn` string, 
  `cn` string, 
  `ad` map, 
  `mi` array)
PARTITIONED BY ( 
  `br` string)
CLUSTERED BY ( 
  bn) 
INTO 2 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
TBLPROPERTIES (
  'bucketing_version'='2');

CREATE TABLE `testj1`(
  `id` int, 
  `can` string, 
  `cn` string, 
  `ad` map, 
  `av` boolean, 
  `mi` array)
PARTITIONED BY ( 
  `brand` string)
CLUSTERED BY ( 
  can) 
INTO 2 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
TBLPROPERTIES (
  'bucketing_version'='2');
{code}
insert some data in both:
{code:java}
insert into testj1 values (100, 'mes_1', 'customer_1',  map('city1', 560077), 
false, array(5, 10), 'brand_1'),
(101, 'mes_2', 'customer_2',  map('city2', 560078), true, array(10, 20), 
'brand_2'),
(102, 'mes_3', 'customer_3',  map('city3', 560079), false, array(15, 30), 
'brand_3'),
(103, 'mes_4', 'customer_4',  map('city4', 560080), true, array(20, 40), 
'brand_4'),
(104, 'mes_5', 'customer_5',  map('city5', 560081), false, array(25, 50), 
'brand_5');

insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0', 
560076),array(0, 0, 0), 'tv'),
(101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
(102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
(103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
(104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
{code}
Do a join between them:
{code:java}
select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2 
on (t1.id = t2.id) order by t1.id;
{code}
Observed results:

!image-2019-11-28-20-46-25-432.png|width=524,height=100!

In the plan, I can see a map join. Disabling it gives the correct result.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 71844: HIVE-22554: ACID: Wait timeout for blocking compaction should be configurable

2019-11-28 Thread Laszlo Pinter via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71844/
---

Review request for hive and Peter Vary.


Repository: hive-git


Description
---

HIVE-22554: ACID: Wait timeout for blocking compaction should be configurable


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
4393a2825e1f465781fc07a6678ebaa2bab906bd 
  
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/AlterTableCompactOperation.java
 fd0ae3a3df731aa690d024dfdbf89f7754ca2a41 


Diff: https://reviews.apache.org/r/71844/diff/1/


Testing
---


Thanks,

Laszlo Pinter

Re: Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

2019-11-28 Thread Denys Kuzmenko via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71812/#review218847
---




ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java
Line 132 (original), 132 (patched)


Redundant


- Denys Kuzmenko


On Nov. 25, 2019, 12:18 p.m., Laszlo Pinter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71812/
> ---
> 
> (Updated Nov. 25, 2019, 12:18 p.m.)
> 
> 
> Review request for hive, Denys Kuzmenko and Peter Vary.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-22534: ACID: Improve Compactor thread logging
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
> ee2c0f3e23ed716f3de0a2740a96a7ec39251bc2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java 
> 10681c0202a32c338e58b3e2eede03657a00774f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java
>  f7e0a85c1f595bb4f112aa051779db3f00c8e572 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java 
> 80119de22f602d9e3cb7a1f60b48e05a37c6a047 
>   
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java
>  41cb4b64fbc79dcf81919769c567b26a2e18cfe5 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 
> 3270175a80992e0efb1e0bfd1f33ffd8a96fcf87 
> 
> 
> Diff: https://reviews.apache.org/r/71812/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Pinter
> 
>

Re: Review Request 71811: Extract Compiler from Driver

2019-11-28 Thread Zoltan Haindrich

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71811/#review218846
---

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 101 (patched)

it would be great to reduce the class tangle between Driver and this 
Compiler class - not sure if that's possible in this patch or not...

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 106 (patched)

nit: can we have this as an apidoc instead of some comments?

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 125 (patched)

I think `plan` should be a local variable; and this method could return it; 
so that compile produces something :)
...and similarily for other fields could probably be made local variables - 
local variables make it more clear what's happening (to me at least); and they 
also help in making the code "drop-to-frame" friendly - which is very handy 
during debugging sessions...

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 132 (patched)

I haven't checked it; this might be something which was even there before:

* parse() throws ParseError (which is not cpe)
* before returning with the exception it sets parseError boolean
* handleException sets compileError unconditionally
* cleanUp does some conditional on parseError; and uses compileError - but 
that is true in that case unconditionally

these parseError/compileError booleans seem to be bad to me...

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 161 (patched)

this doesn't belong here; I think it is safeto leave this reset thing in 
the Driver; and not pollute this new class with the "resetTaskIds" thing; or 
there is something I'm not considering?

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 165 (patched)

I think DriverState should live at the "Driver" level; and not get mixed 
into other classes (followup?)

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 167 (patched)

I don't feel this closely connected to compilation; queryId could be 
assigned in the driver (followup?)

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 188 (patched)

okay...after I've made a few comments in this method right now I kinda feel 
that we should try to leave this "initialize" outside in the Driver somewhere; 
it kind works only on the "driverContext" anyway...

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 193 (patched)

I don't think this should be here; can't we have a context when we enter 
the compiler?

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 498 (patched)

I think it might make sense to relocate this method into the `ExplainTask` 
itself; instead of keeping it here...

ql/src/java/org/apache/hadoop/hive/ql/Compiler.java
Lines 565 (patched)

the method `Driver.dumpMetaCallTimingWithoutEx` can be made static; however 
even with that modifier I'm not sure where to put itright now it doesn't 
seem to belong to neither of Driver/Compiler class

- Zoltan Haindrich

On Nov. 25, 2019, 10:24 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71811/
> ---
> 
> (Updated Nov. 25, 2019, 10:24 a.m.)
> 
> 
> Review request for hive and Zoltan Haindrich.
> 
> 
> Bugs: HIVE-22526
> https://issues.apache.org/jira/browse/HIVE-22526
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/Compiler.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java bb41c15bb4 
>

[jira] [Created] (HIVE-22560) ACID MERGE fails if NOT NULL constraint is used for column and PDD is enabled

2019-11-28 Thread Aron Hamvas (Jira)

Aron Hamvas created HIVE-22560:
--

 Summary: ACID MERGE fails if NOT NULL constraint is used for 
column and PDD is enabled
 Key: HIVE-22560
 URL: https://issues.apache.org/jira/browse/HIVE-22560
 Project: Hive
  Issue Type: Bug
  Components: Query Planning
Affects Versions: 3.1.2
Reporter: Aron Hamvas


If hive.optimize.ppd=true is set, ACID MERGE can fail if the column used in 
matching has NOT NULL constraint.

To reproduce the issue:
{code:sql}
CREATE TABLE SIMPLECDCMASTER (CLI_NUM VARCHAR(13) NOT NULL , FIELD1 
VARCHAR(100));
CREATE TABLE SIMPLECDCDELTA (CLI_NUM VARCHAR(13), FIELD1 VARCHAR(100));

insert into simplecdcdelta values (123, 'abc');

merge into simplecdcmaster as m using simplecdcdelta as d on m.cli_num = 
d.cli_num
when matched then update set field1=d.field1
when not matched then insert values (d.cli_num, d.field1);
{code}

The NOT NULL constrained is then enforced in the Filter operator processing the 
output of the right outer join, where, of course, the not matched records have 
nulls in all columns.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71821: HIVE-22544

2019-11-28 Thread Krisztian Kasa


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71821/
---

(Updated Nov. 28, 2019, 8:51 a.m.)


Review request for hive and Jesús Camacho Rodríguez.


Bugs: HIVE-22544
https://issues.apache.org/jira/browse/HIVE-22544


Repository: hive-git


Description
---

Disable null sort order at user level


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 77c8300e32 
  ql/src/java/org/apache/hadoop/hive/ql/plan/TopNKeyDesc.java dc45800d74 
  ql/src/test/results/clientpositive/llap/constprog_dpp.q.out 11af918604 
  ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out 90dd54b78e 
  ql/src/test/results/clientpositive/llap/cte_mat_3.q.out 2433e99dbb 
  ql/src/test/results/clientpositive/llap/cte_mat_4.q.out 52ed4a78b3 
  ql/src/test/results/clientpositive/llap/cte_mat_5.q.out fac092947a 
  ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out a095fcc0ae 
  ql/src/test/results/clientpositive/llap/dynamic_semijoin_user_level.q.out 
4b9035e5b9 
  ql/src/test/results/clientpositive/llap/empty_join.q.out b19ce14c78 
  ql/src/test/results/clientpositive/llap/estimate_pkfk_filtered_fk.q.out 
b74a20f305 
  ql/src/test/results/clientpositive/llap/estimate_pkfk_nocond.q.out 5a58b13b8c 
  ql/src/test/results/clientpositive/llap/estimate_pkfk_push.q.out 2af0fb5ee3 
  ql/src/test/results/clientpositive/llap/explainanalyze_2.q.out f5b2e4b294 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 735296f814 
  ql/src/test/results/clientpositive/llap/explainuser_2.q.out b90fb55fd1 
  ql/src/test/results/clientpositive/llap/explainuser_4.q.out 3504d17875 
  ql/src/test/results/clientpositive/llap/groupby_groupingset_bug.q.out 
a8c1d56ef7 
  ql/src/test/results/clientpositive/llap/reopt_dpp.q.out 141b2b617d 
  ql/src/test/results/clientpositive/llap/retry_failure_reorder.q.out 
9e1c249ab7 
  ql/src/test/results/clientpositive/llap/retry_failure_stat_changes.q.out 
204fa8f711 
  ql/src/test/results/clientpositive/llap/runtime_stats_hs2.q.out f88aa718f8 
  ql/src/test/results/clientpositive/llap/runtime_stats_merge.q.out a11e81f1bc 
  ql/src/test/results/clientpositive/llap/windowing_gby.q.out 0888fd979e 
  ql/src/test/results/clientpositive/perf/tez/constraints/mv_query44.q.out 
6a67ca6823 
  ql/src/test/results/clientpositive/perf/tez/constraints/query1.q.out 
211e92ecf4 
  ql/src/test/results/clientpositive/perf/tez/constraints/query10.q.out 
7ad55b5c5f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query11.q.out 
2deb5827a5 
  ql/src/test/results/clientpositive/perf/tez/constraints/query12.q.out 
035c908ec3 
  ql/src/test/results/clientpositive/perf/tez/constraints/query13.q.out 
b27552f901 
  ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out 
9b07ceb3a7 
  ql/src/test/results/clientpositive/perf/tez/constraints/query15.q.out 
7c0e6cf8cd 
  ql/src/test/results/clientpositive/perf/tez/constraints/query16.q.out 
377307f6e3 
  ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out 
44fd104a9b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query18.q.out 
b65bebc22b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query19.q.out 
d313b21add 
  ql/src/test/results/clientpositive/perf/tez/constraints/query2.q.out 
e1308a315a 
  ql/src/test/results/clientpositive/perf/tez/constraints/query20.q.out 
a10979641a 
  ql/src/test/results/clientpositive/perf/tez/constraints/query21.q.out 
d944adec04 
  ql/src/test/results/clientpositive/perf/tez/constraints/query22.q.out 
0e1f13e65e 
  ql/src/test/results/clientpositive/perf/tez/constraints/query23.q.out 
761369c88b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query24.q.out 
8b4fea7d80 
  ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out 
1ca55ca290 
  ql/src/test/results/clientpositive/perf/tez/constraints/query26.q.out 
90a56acafc 
  ql/src/test/results/clientpositive/perf/tez/constraints/query27.q.out 
58b2b86cdf 
  ql/src/test/results/clientpositive/perf/tez/constraints/query28.q.out 
9634957a52 
  ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out 
073ae52f1f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query3.q.out 
50beff8772 
  ql/src/test/results/clientpositive/perf/tez/constraints/query30.q.out 
6aa98f2bdd 
  ql/src/test/results/clientpositive/perf/tez/constraints/query31.q.out 
3f0e054d0c 
  ql/src/test/results/clientpositive/perf/tez/constraints/query32.q.out 
f3f381a26e 
  ql/src/test/results/clientpositive/perf/tez/constraints/query33.q.out 
fbab5700b7 
  ql/src/test/results/clientpositive/perf/tez/constraints/query34.q.out 
8f730c8f3a 
  ql/src/test/results/clientpositive/perf/tez/constraints/query35.q.out 
4ca9d3cb61 
  ql/src/test/results/clientpositive/perf/tez/constraints/query36.q.out 
d8fa6f4f5a

[jira] [Created] (HIVE-22559) Maintain ownership of parent directories of an external table directory after replication

2019-11-28 Thread Ashutosh Bapat (Jira)

Ashutosh Bapat created HIVE-22559:
-

 Summary: Maintain ownership of parent directories of an external 
table directory after replication
 Key: HIVE-22559
 URL: https://issues.apache.org/jira/browse/HIVE-22559
 Project: Hive
  Issue Type: Improvement
Reporter: Ashutosh Bapat
Assignee: Anishek Agarwal


For replicating an external table we specify a base directory on the target 
(say /base_ext for example). The path of an external table directory on the 
source (say /xyz/abc/ext_t1) is prefixed with the base directory on the target 
(/base_ext in our example) when replicating the external table data. Thus the 
path of the external table on the target becomes /base_ext/xyz/abc/ext_t1. In 
this path only the ownership permissions of ext_t1 directory is preserved but 
the owenship of xyz and abc directories is set to the user executing REPL LOAD. 
Instead we should preserve the ownership of xyz and abc as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71820: HIVE-20150

2019-11-28 Thread Krisztian Kasa


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71820/
---

(Updated Nov. 28, 2019, 8 a.m.)


Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich.


Bugs: HIVE-20150
https://issues.apache.org/jira/browse/HIVE-20150


Repository: hive-git


Description
---

TopNKey pushdown

1. Apply patch: 
https://issues.apache.org/jira/secure/attachment/12941630/HIVE-20150.11.patch
2. TopNKey introduction depends only from Reduce Sink with topn property >= 0
3. Implement TopNKey operator pushdown through: projection, group by, redeuce 
sink, left outer join, other topnkey
4. Add sort order and null sort order direction check when determining if the 
topnkey op can be pushed
5. Implement handling cases when topnkey op and the parent op has a common key 
prefix only.
6. turn off topnkey optimization by default


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 4393a2825e 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/CommonKeyPrefix.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java 
4b4cf99503 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyPushdownProcessor.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java bf58bd8bb8 
  ql/src/test/org/apache/hadoop/hive/ql/optimizer/TestCommonKeyPrefix.java 
PRE-CREATION 
  ql/src/test/queries/clientpositive/topnkey.q 057b6a45ba 
  ql/src/test/queries/clientpositive/vector_topnkey.q 85c5880cd6 
  ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 0c051c926b 
  ql/src/test/results/clientpositive/llap/check_constraint.q.out 9f2c9a1cd0 
  ql/src/test/results/clientpositive/llap/constraints_optimization.q.out 
b6d210becf 
  ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 
9343e078b7 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 735296f814 
  ql/src/test/results/clientpositive/llap/explainuser_2.q.out b90fb55fd1 
  ql/src/test/results/clientpositive/llap/external_jdbc_table_perf.q.out 
545cce75a9 
  ql/src/test/results/clientpositive/llap/filter_union.q.out 0df77762a0 
  ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 3fdd77d802 
  ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out efa8c38d7c 
  ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out 
ffe5f6fb22 
  ql/src/test/results/clientpositive/llap/offset_limit.q.out 23f2de46e5 
  ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 
4ecb7bc46d 
  ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out 
0eac389eb7 
  
ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out
 4362fb6f2e 
  ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out 
24468c9a1b 
  
ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out 
45890a1890 
  ql/src/test/results/clientpositive/llap/semijoin_reddedup.q.out 0e9723b8f3 
  ql/src/test/results/clientpositive/llap/subquery_ALL.q.out d910c1a79d 
  ql/src/test/results/clientpositive/llap/subquery_ANY.q.out 91472d631e 
  ql/src/test/results/clientpositive/llap/topnkey.q.out 1e77587f82 
  ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out cc2dc47280 
  ql/src/test/results/clientpositive/llap/vector_char_2.q.out f7e76e5a8b 
  
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
 6fd15e7101 
  ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out 
d6325982e3 
  ql/src/test/results/clientpositive/llap/vector_mr_diff_schema_alias.q.out 
4d417b9c3d 
  ql/src/test/results/clientpositive/llap/vector_reduce_groupby_decimal.q.out 
97a211cfc6 
  ql/src/test/results/clientpositive/llap/vector_string_concat.q.out a8019be7aa 
  ql/src/test/results/clientpositive/llap/vector_topnkey.q.out c140bdfd37 
  ql/src/test/results/clientpositive/llap/vectorization_limit.q.out 7326adf522 
  ql/src/test/results/clientpositive/perf/tez/cbo_query14.q.out e9308cd709 
  ql/src/test/results/clientpositive/perf/tez/cbo_query77.q.out 02caf99f7d 
  ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out 
43e1b2b5c2 
  ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query77.q.out 
2f75361df1 
  ql/src/test/results/clientpositive/perf/tez/constraints/query10.q.out 
7ad55b5c5f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out 
9b07ceb3a7 
  ql/src/test/results/clientpositive/perf/tez/constraints/query15.q.out 
7c0e6cf8cd 
  ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out 
44fd104a9b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out 
1ca55ca290 
  ql/src/test/results/clientpositive/perf/tez/constraints/query26.q.out 
90a56acafc

Re: Review Request 71821: HIVE-22544

[jira] [Created] (HIVE-22563) Required field 'client_protocol' is unset (hive server backward compatibility)

[jira] [Created] (HIVE-22562) Harmonize SessionState.getUserName

[jira] [Created] (HIVE-22561) Data loss on map join for bucketed, partitioned table

Review Request 71844: HIVE-22554: ACID: Wait timeout for blocking compaction should be configurable

Re: Review Request 71812: HIVE-22534: ACID: Improve Compactor thread logging

Re: Review Request 71811: Extract Compiler from Driver

[jira] [Created] (HIVE-22560) ACID MERGE fails if NOT NULL constraint is used for column and PDD is enabled

Re: Review Request 71821: HIVE-22544

[jira] [Created] (HIVE-22559) Maintain ownership of parent directories of an external table directory after replication

Re: Review Request 71820: HIVE-20150

11 matches

Site Navigation

Mail list logo

Footer information