[jira] [Created] (HIVE-21190) hive thrift server may be blocked by session level waiting!

[kongxianghe (JIRA)]

kongxianghe created HIVE-21190:
--

 Summary: hive thrift server may be blocked by session level 
waiting!
 Key: HIVE-21190
 URL: https://issues.apache.org/jira/browse/HIVE-21190
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
 Environment: hdp ambari 26

hive1.2.0
Reporter: kongxianghe
Assignee: kongxianghe


# cause by an error UDF function！time_waiting(Long sleepSeconds)
{code}
public class UDFTimeWaiting extends UDF throws Exception{
  public String evaluate(Long sleepSeconds){
 ...
 Thread.sleep(Long.parseLong(sleepSeconds) * 1000);
 return "ok";
 }
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-21189) hive.merge.nway.joins should default to false

[Ashutosh Chauhan (JIRA)]

Ashutosh Chauhan created HIVE-21189:
---

 Summary: hive.merge.nway.joins should default to false
 Key: HIVE-21189
 URL: https://issues.apache.org/jira/browse/HIVE-21189
 Project: Hive
  Issue Type: Improvement
  Components: Configuration
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-21188) SemanticAnalyzerException for query on view with masked table

[Jesus Camacho Rodriguez (JIRA)]

Jesus Camacho Rodriguez created HIVE-21188:
--

 Summary: SemanticAnalyzerException for query on view with masked 
table
 Key: HIVE-21188
 URL: https://issues.apache.org/jira/browse/HIVE-21188
 Project: Hive
  Issue Type: Bug
  Components: Parser
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


When view reference is fully qualified. Following q file can be used to 
reproduce the issue:

{code}
--! qt:dataset:srcpart
--! qt:dataset:src
set hive.mapred.mode=nonstrict;
set 
hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactoryForTest;

create database atlasmask;
use atlasmask;
create table masking_test_n8 (key int, value int);
insert into masking_test_n8 values(1,1), (2,2);
create view testv(c,d) as select * from masking_test_n8;

select * from `atlasmask`.`testv`;
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [NOTICE] Mandatory migration of git repos to gitbox.apache.org - one week left!

[Jesus Camacho Rodriguez]

The repository has been moved.

https://gitbox.apache.org/repos/asf?p=hive.git

-Jesús

On 1/30/19, 10:26 AM, "Jesus Camacho Rodriguez" 
 wrote:

As we had come to a consensus, I have created 
https://issues.apache.org/jira/browse/INFRA-17785 to migrate to gitbox.

-Jesús



On 1/30/19, 12:10 AM, "Apache Infrastructure Team" 
 wrote:

Hello again, hive folks.
This is a reminder that you have *one week left* before the mandatory
mass-migration from git-wip-us to gitbox.

As stated earlier in 2018, and reiterated a few times, all git
repositories must be migrated from the git-wip-us.apache.org URL to
gitbox.apache.org, as the old service is being decommissioned. Your
project is receiving this email because you still have repositories on
git-wip-us that needs to be migrated.

The following repositories on git-wip-us belong to your project:
 - hive.git


We are now entering the remaining one week of the mandated
(coordinated) move stage of the roadmap, and you are asked to please
coordinate migration with the Apache Infrastructure Team before February
7th. All repositories not migrated on February 7th will be mass migrated
without warning, and we'd appreciate it if we could work together to
avoid a big mess that day :-).

As stated earlier, moving to gitbox means you will get full write access
on GitHub as well, and be able to close/merge pull requests and much
more. The move is mandatory for all Apache projects using git.

To have your repositories moved, please follow these steps:

- Ensure consensus on the move (a link to a lists.apache.org thread will
  suffice for us as evidence).
- Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA

Your migration should only take a few minutes. If you wish to migrate
at a specific time of day or date, please do let us know in the ticket,
otherwise we will migrate at the earliest convenient time.

There will be redirects in place from git-wip to gitbox, so requests
using the old remote origins should still work (however we encourage
people to update their remotes once migration has completed).

As always, we appreciate your understanding and patience as we move
things around and work to provide better services and features for
the Apache Family.

Should you wish to contact us with feedback or questions, please do so
at: us...@infra.apache.org.


With regards,
Apache Infrastructure

Re: [NOTICE] Mandatory migration of git repos to gitbox.apache.org - one week left!

[Jesus Camacho Rodriguez]

As we had come to a consensus, I have created 
https://issues.apache.org/jira/browse/INFRA-17785 to migrate to gitbox.

-Jesús



On 1/30/19, 12:10 AM, "Apache Infrastructure Team"  
wrote:

Hello again, hive folks.
This is a reminder that you have *one week left* before the mandatory
mass-migration from git-wip-us to gitbox.

As stated earlier in 2018, and reiterated a few times, all git
repositories must be migrated from the git-wip-us.apache.org URL to
gitbox.apache.org, as the old service is being decommissioned. Your
project is receiving this email because you still have repositories on
git-wip-us that needs to be migrated.

The following repositories on git-wip-us belong to your project:
 - hive.git


We are now entering the remaining one week of the mandated
(coordinated) move stage of the roadmap, and you are asked to please
coordinate migration with the Apache Infrastructure Team before February
7th. All repositories not migrated on February 7th will be mass migrated
without warning, and we'd appreciate it if we could work together to
avoid a big mess that day :-).

As stated earlier, moving to gitbox means you will get full write access
on GitHub as well, and be able to close/merge pull requests and much
more. The move is mandatory for all Apache projects using git.

To have your repositories moved, please follow these steps:

- Ensure consensus on the move (a link to a lists.apache.org thread will
  suffice for us as evidence).
- Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA

Your migration should only take a few minutes. If you wish to migrate
at a specific time of day or date, please do let us know in the ticket,
otherwise we will migrate at the earliest convenient time.

There will be redirects in place from git-wip to gitbox, so requests
using the old remote origins should still work (however we encourage
people to update their remotes once migration has completed).

As always, we appreciate your understanding and patience as we move
things around and work to provide better services and features for
the Apache Family.

Should you wish to contact us with feedback or questions, please do so
at: us...@infra.apache.org.


With regards,
Apache Infrastructure

Please unsubscribe me

[Daniel Takacs]

I tried to unsubscribe several times but still it is not working.  Please help 
me unsubscribe 
Sent from my iPhone

Re: Review Request 69830: HIVE-21143 Add rewrite rules to open/close Between operators

[Zoltan Haindrich]

> On Jan. 24, 2019, 7:53 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java
> > Lines 47 (patched)
> > 
> >
> > Instead of using the shaded version, we should use our own guava 
> > version directly.

I was sure that it will be available...but it seems like these graph classes 
are added in Guava 20.0; but Hive uses 19

In the latest patch I'm considering to upgrade to Guava 23 - because thats what 
HBase uses
I've seen HIVE-15393; so not sure if that would be a viable option or not - and 
that should be probably done in a separate ticket as it might involve hadoop or 
other libraries

I ended up "rolling some simple graph class" to make this work...


> On Jan. 24, 2019, 7:53 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/test/results/clientpositive/join34.q.out
> > Line 35 (original)
> > 
> >
> > We lost the optimized SQL, maybe because we do not support unparsing of 
> > BETWEEN clause? We can tackle in follow-up, but it would be good to know 
> > the root cause.

yes, opened HIVE-21187


> On Jan. 24, 2019, 7:53 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/test/results/clientpositive/vector_between_columns.q.out
> > Line 160 (original), 160 (patched)
> > 
> >
> > Nothing changed below in the plan, but BETWEEN expression columns have 
> > changed here. Is this correct?

yes this is correct:
```
_col4 <= _col1 and _col1 <= _col4  (_col4 in _col1 and _col1)
# change the order of the operands
_col1 <= _col4 and _col4 <= _col1  (_col1 in _col4 and _col4)
```

the best simplification would be to notice that in this case `_col1 = _col4`; 
we miss that in `RexSimplify`

I've seen a ticket from Gopal: https://issues.apache.org/jira/browse/HIVE-18167
and probably not closing BETWEEN in case the range has column references might 
be a way to address that


- Zoltan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69830/#review212278
---


On Jan. 24, 2019, 3:31 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69830/
> ---
> 
> (Updated Jan. 24, 2019, 3:31 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and 
> Vineet Garg.
> 
> 
> Bugs: HIVE-21143
> https://issues.apache.org/jira/browse/HIVE-21143
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> patch #1 here is #7 on jira
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java
>  9a525adeb9 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
>  a920e4096e 
>   ql/src/test/queries/clientpositive/udf_between.q 9ee33520b9 
>   ql/src/test/results/clientpositive/correlationoptimizer8.q.out 9ad55cd91f 
>   ql/src/test/results/clientpositive/druid/druidmini_test_ts.q.out 9c412d97dd 
>   ql/src/test/results/clientpositive/druid_timeseries.q.out fd69771e61 
>   ql/src/test/results/clientpositive/filter_numeric.q.out 74ce762191 
>   ql/src/test/results/clientpositive/infer_join_preds.q.out d73005c82d 
>   ql/src/test/results/clientpositive/join34.q.out f5a280f186 
>   ql/src/test/results/clientpositive/join35.q.out 227782126c 
>   ql/src/test/results/clientpositive/join45.q.out 8a6fd8fcec 
>   ql/src/test/results/clientpositive/join47.q.out 475079dacf 
>   ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out 
> d98a9b665d 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out 
> abefa7eaa2 
>   ql/src/test/results/clientpositive/llap/subquery_scalar.q.out 2a684018df 
>   ql/src/test/results/clientpositive/llap/vector_between_columns.q.out 
> a91a36b358 
>   ql/src/test/results/clientpositive/llap/vector_interval_2.q.out 3b2b7ba971 
>   ql/src/test/results/clientpositive/mapjoin47.q.out 67b2f4cd03 
>   ql/src/test/results/clientpositive/partition_wise_fileformat2.q.out 
> a1c5ab3071 
>   ql/src/test/results/clientpositive/pcr.q.out 660556ffe9 
>   ql/src/test/results/clientpositive/perf/spark/query13.q.out 2b6c19d6c2 
>   ql/src/test/results/clientpositive/perf/spark/query21.q.out 6c2efe75c9 
>   ql/src/test/results/clientpositive/perf/spark/query34.q.out 1dd58b399f 
>   ql/src/test/results/clientpositive/perf/spark/query48.q.out 024d205290 
>   ql/src/test/results/clientpositive/perf/spark/query54.q.out 690f1d4cdf 
>   ql/src/test/results/clientpositive/perf/spark/query58.q.out 

[jira] [Created] (HIVE-21187) OptimizedSql is not shown when the expression contains BETWEENs

[Zoltan Haindrich (JIRA)]

Zoltan Haindrich created HIVE-21187:
---

 Summary: OptimizedSql is not shown when the expression contains 
BETWEENs
 Key: HIVE-21187
 URL: https://issues.apache.org/jira/browse/HIVE-21187
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


in the patch for HIVE-21143 we see that a lot of optimized sql printouts are 
going away because of this



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] hive pull request #524: HIVE-21186: External tables replication throws NPE i...

[sankarh]

GitHub user sankarh opened a pull request:

https://github.com/apache/hive/pull/524

HIVE-21186: External tables replication throws NPE if 
hive.repl.replica.external.table.base.dir is not fully qualified HDFS path.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sankarh/hive HIVE-21186

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/524.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #524


commit 06653ca2d5a607784d7c0ce47d7c5b28769e9a02
Author: Sankar Hariappan 
Date:   2019-01-30T09:12:02Z

HIVE-21186: External tables replication throws NPE if 
hive.repl.replica.external.table.base.dir is not fully qualified HDFS path.




---

[NOTICE] Mandatory migration of git repos to gitbox.apache.org - one week left!

[Apache Infrastructure Team]

Hello again, hive folks.
This is a reminder that you have *one week left* before the mandatory
mass-migration from git-wip-us to gitbox.

As stated earlier in 2018, and reiterated a few times, all git
repositories must be migrated from the git-wip-us.apache.org URL to
gitbox.apache.org, as the old service is being decommissioned. Your
project is receiving this email because you still have repositories on
git-wip-us that needs to be migrated.

The following repositories on git-wip-us belong to your project:
 - hive.git


We are now entering the remaining one week of the mandated
(coordinated) move stage of the roadmap, and you are asked to please
coordinate migration with the Apache Infrastructure Team before February
7th. All repositories not migrated on February 7th will be mass migrated
without warning, and we'd appreciate it if we could work together to
avoid a big mess that day :-).

As stated earlier, moving to gitbox means you will get full write access
on GitHub as well, and be able to close/merge pull requests and much
more. The move is mandatory for all Apache projects using git.

To have your repositories moved, please follow these steps:

- Ensure consensus on the move (a link to a lists.apache.org thread will
  suffice for us as evidence).
- Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA

Your migration should only take a few minutes. If you wish to migrate
at a specific time of day or date, please do let us know in the ticket,
otherwise we will migrate at the earliest convenient time.

There will be redirects in place from git-wip to gitbox, so requests
using the old remote origins should still work (however we encourage
people to update their remotes once migration has completed).

As always, we appreciate your understanding and patience as we move
things around and work to provide better services and features for
the Apache Family.

Should you wish to contact us with feedback or questions, please do so
at: us...@infra.apache.org.


With regards,
Apache Infrastructure

[jira] [Created] (HIVE-21186) REPL LOAD for external tables throws NPE if relative path is set for hive.repl.replica.external.table.base.dir.

[Sankar Hariappan (JIRA)]

Sankar Hariappan created HIVE-21186:
---

 Summary: REPL LOAD for external tables throws NPE  if relative 
path is set for hive.repl.replica.external.table.base.dir.
 Key: HIVE-21186
 URL: https://issues.apache.org/jira/browse/HIVE-21186
 Project: Hive
  Issue Type: Bug
  Components: repl
Affects Versions: 4.0.0
Reporter: Sankar Hariappan
Assignee: Sankar Hariappan


REPL DUMP is fine. Load seems to be throwing exception:
{code}
2019-01-29 09:25:12,671 ERROR HiveServer2-Background-Pool: Thread-4864: 
ql.Driver (SessionState.java:printError(1129)) - FAILED: Execution Error, 
return code 4 from org.apache.hadoop.hive.ql.exec.repl.ReplLoadTask. 
java.lang.NullPointerException
2019-01-29 09:25:12,671 INFO HiveServer2-Background-Pool: Thread-4864: 
ql.Driver (Driver.java:execute(1661)) - task failed with
org.apache.hadoop.hive.ql.parse.SemanticException: 
java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.repl.bootstrap.load.table.LoadTable.tasks(LoadTable.java:154)
at 
org.apache.hadoop.hive.ql.exec.repl.ReplLoadTask.executeBootStrapLoad(ReplLoadTask.java:141)
at 
org.apache.hadoop.hive.ql.exec.repl.ReplLoadTask.execute(ReplLoadTask.java:82)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:177)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:93)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1777)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1511)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1308)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1175)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
at 
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
at 
org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
at 
org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at 
org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:273)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.repl.bootstrap.load.util.PathUtils.getExternalTmpPath(PathUtils.java:35)
at 
org.apache.hadoop.hive.ql.exec.repl.bootstrap.load.table.LoadTable.loadTableTask(LoadTable.java:245)
at 
org.apache.hadoop.hive.ql.exec.repl.bootstrap.load.table.LoadTable.newTableTasks(LoadTable.java:189)
at 
org.apache.hadoop.hive.ql.exec.repl.bootstrap.load.table.LoadTable.tasks(LoadTable.java:136)
... 23 more
{code}
REPL Load statement: 
{code}
REPL LOAD `testdb1_tgt` FROM 
'hdfs://ctr-e139-1542663976389-56533-01-11.hwx.site:8020/apps/hive/repl/c9476207-8179-4db7-b947-ba67c950a340'
 WITH 
('hive.query.id'='testHive1_3dd5e281-89ef-4054-850e-8a34386fc2c8','hive.exec.parallel'='true','hive.repl.replica.external.table.base.dir'='/tmp/someNewloc/','hive.repl.include.external.tables'='true','mapreduce.map.java.opts'='-Xmx640m','hive.distcp.privileged.doAs'='beacon','distcp.options.pugpb'='')
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)