[jira] [Created] (HIVE-15939) Make cast expressions comply more to sql2011

2017-02-15 Thread Zoltan Haindrich (JIRA) 
Zoltan Haindrich created HIVE-15939:
---

 Summary: Make cast expressions comply more to sql2011
 Key: HIVE-15939
 URL: https://issues.apache.org/jira/browse/HIVE-15939
 Project: Hive
  Issue Type: Sub-task
Reporter: Zoltan Haindrich


in HIVE-6590 Jason have uncovered the fact that UDFToBoolean treats all 
non-empty strings as true.

It would be great to have the cast expressions closer to the standard...at 
least when there is an expected behaviour from the user;

like {{cast('false' as boolean)}} should be true.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] hive pull request #150: mysql schema script should add a comma.

2017-02-15 Thread hellopower 
GitHub user hellopower opened a pull request:

https://github.com/apache/hive/pull/150

mysql schema script should add a comma.

when I run command  'schematool -dbType mysql -initSchema -verbose'.
error:
 You have an error in your SQL syntax; check the manual that corresponds to 
your MySQL server version for the right syntax to use near ') ENGINE=InnoDB 
DEFAULT CHARSET=latin1' at line 1

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hellopower/hive master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/150.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #150


commit 700701e1fb0b0314e7e856954a35fa0b94fceb5a
Author: wubiao 
Date:   2017-02-16T03:39:55Z

mysql schema script should add a comma.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-15938) position alias in order by fails for union queries

2017-02-15 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-15938:
---

 Summary: position alias in order by fails for union queries
 Key: HIVE-15938
 URL: https://issues.apache.org/jira/browse/HIVE-15938
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15937) RetryingMetaStoreClient.invoke() should retry Trift exception even if not wrapped in MetaException

2017-02-15 Thread Eugene Koifman (JIRA) 
Eugene Koifman created HIVE-15937:
-

 Summary: RetryingMetaStoreClient.invoke() should retry Trift 
exception even if not wrapped in MetaException
 Key: HIVE-15937
 URL: https://issues.apache.org/jira/browse/HIVE-15937
 Project: Hive
  Issue Type: Bug
  Components: Metastore, Transactions
Reporter: Eugene Koifman
Assignee: Eugene Koifman






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15936) ConcurrentModificationException in ATSHook

2017-02-15 Thread Daniel Dai (JIRA) 
Daniel Dai created HIVE-15936:
-

 Summary: ConcurrentModificationException in ATSHook
 Key: HIVE-15936
 URL: https://issues.apache.org/jira/browse/HIVE-15936
 Project: Hive
  Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
 Attachments: HIVE-15936.1.patch

See ATSHook error:

{noformat}
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) 
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1471) 
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1469) 
~[?:1.8.0_112]
at java.util.AbstractCollection.toArray(AbstractCollection.java:196) 
~[?:1.8.0_112]
at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:290) 
~[guava-14.0.1.jar:?]
at 
org.apache.hadoop.hive.ql.log.PerfLogger.getEndTimes(PerfLogger.java:219) 
~[hive-common-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at 
org.apache.hadoop.hive.ql.hooks.ATSHook.createPostHookEvent(ATSHook.java:347) 
~[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at org.apache.hadoop.hive.ql.hooks.ATSHook$2.run(ATSHook.java:206) 
[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[?:1.8.0_112]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[?:1.8.0_112]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[?:1.8.0_112]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{noformat}

According to [~jdere], ATSHook is currently accessing the PerfLogger on a 
separate thread, which means the main query thread can potentially write to the 
PerfLogger at the same time.
The ATSHook should access the PerfLogger on the main query thread, before it 
sends the execution to the ATS Logger thread.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15935) ACL is not set in ATS data

2017-02-15 Thread Daniel Dai (JIRA) 
Daniel Dai created HIVE-15935:
-

 Summary: ACL is not set in ATS data
 Key: HIVE-15935
 URL: https://issues.apache.org/jira/browse/HIVE-15935
 Project: Hive
  Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai


When publishing ATS info, Hive does not set ACL, that make Hive ATS entries 
visible to all users. On the other hand, Tez ATS entires is using Tez DAG ACL 
which limit both view/modify ACL to end user only. We shall make them 
consistent. In the Jira, I am going to limit ACL to end user for both Tez ATS 
and Hive ATS, also provide config "hive.view.acls" and "hive.modify.acls" if 
user need to overridden.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15934) Downgrade Maven surefire plugin from 2.19.1 to 2.18.1

2017-02-15 Thread Wei Zheng (JIRA) 
Wei Zheng created HIVE-15934:


 Summary: Downgrade Maven surefire plugin from 2.19.1 to 2.18.1
 Key: HIVE-15934
 URL: https://issues.apache.org/jira/browse/HIVE-15934
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Wei Zheng
Assignee: Wei Zheng


Surefire 2.19.1 has some issue 
(https://issues.apache.org/jira/browse/SUREFIRE-1255) which caused debugging 
session to abort after a short period of time. Many IntelliJ users have seen 
this, although it looks fine for Eclipse users. Version 2.18.1 works fine.

We'd better make the change to not impact the development for IntelliJ guys. We 
can upgrade again once the root cause is figured out.

cc [~kgyrtkirk] [~ashutoshc]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15933) Improve plans for correlated subquery with join and predicate

2017-02-15 Thread Vineet Garg (JIRA) 
Vineet Garg created HIVE-15933:
--

 Summary: Improve plans for correlated subquery with join and 
predicate
 Key: HIVE-15933
 URL: https://issues.apache.org/jira/browse/HIVE-15933
 Project: Hive
  Issue Type: Sub-task
  Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg


This is a continuation of HIVE-15905
for queries such as:

{code:SQL}
explain select  
  cd_gender,
  cd_marital_status,
  cd_education_status,
  count(*) cnt1,
  cd_purchase_estimate,
  count(*) cnt2,
  cd_credit_rating,
  count(*) cnt3,
  cd_dep_count,
  count(*) cnt4,
  cd_dep_employed_count,
  count(*) cnt5,
  cd_dep_college_count,
  count(*) cnt6
 from
  customer c,customer_address ca,customer_demographics
 where
  c.c_current_addr_sk = ca.ca_address_sk and
  ca_county in ('Walker County','Richland County','Gaines County','Douglas 
County','Dona Ana County') and
  cd_demo_sk = c.c_current_cdemo_sk and 
  exists (select *
  from store_sales,date_dim
  where c.c_customer_sk = ss_customer_sk and
ss_sold_date_sk = d_date_sk and
d_year = 2002 and
d_moy between 4 and 4+3)
 group by cd_gender,
  cd_marital_status,
  cd_education_status,
  cd_purchase_estimate,
  cd_credit_rating,
  cd_dep_count,
  cd_dep_employed_count,
  cd_dep_college_count
 order by cd_gender,
  cd_marital_status,
  cd_education_status,
  cd_purchase_estimate,
  cd_credit_rating,
  cd_dep_count,
  cd_dep_employed_count,
  cd_dep_college_count
limit 100;
{code}

HIVE generates un-necessary joins to produce value for correlated columns.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] hive pull request #149: HIVE-15929. Fix HiveDecimalWritable compatibility wi...

2017-02-15 Thread omalley 
GitHub user omalley opened a pull request:

https://github.com/apache/hive/pull/149

HIVE-15929. Fix HiveDecimalWritable compatibility with Hive 2.1.

This allows Hive 2.1 to work without getting an exception.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/omalley/hive hive-15929

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/149.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #149


commit aeaf8dc2d8ef0adf0c6fdf216da89453a807459a
Author: Owen O'Malley 
Date:   2017-02-15T01:25:46Z

HIVE-15929. Fix HiveDecimalWritable compatibility with Hive 2.1.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-15932) add "explain ast"

2017-02-15 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-15932:
---

 Summary: add "explain ast"
 Key: HIVE-15932
 URL: https://issues.apache.org/jira/browse/HIVE-15932
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin


AST was removed in explain extended in HIVE-13533; that makes sense from users 
perspective for the common case, but it would be useful for Hive developers and 
advanced users to see the AST to diagnose issues. "Explain ast" command can be 
added to dump the AST.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15931) JDBC: Improve logging when using ZooKeeper

2017-02-15 Thread Vaibhav Gumashta (JIRA) 
Vaibhav Gumashta created HIVE-15931:
---

 Summary: JDBC: Improve logging when using ZooKeeper
 Key: HIVE-15931
 URL: https://issues.apache.org/jira/browse/HIVE-15931
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 2.2.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15929) Fix HiveDecimalWritable

2017-02-15 Thread Owen O'Malley (JIRA) 
Owen O'Malley created HIVE-15929:


 Summary: Fix HiveDecimalWritable 
 Key: HIVE-15929
 URL: https://issues.apache.org/jira/browse/HIVE-15929
 Project: Hive
  Issue Type: Bug
Reporter: Owen O'Malley


HIVE-15335 broke compatibility with Hive 2.1 by making 
HiveDecimalWritable.getInternalStorate() throw an exception when called on an 
unset value. It is easy to instead return an empty array, which will allow the 
old code to allocate a new array.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15928) Parallelization of Select queries in Druid handler

2017-02-15 Thread Jesus Camacho Rodriguez (JIRA) 
Jesus Camacho Rodriguez created HIVE-15928:
--

 Summary: Parallelization of Select queries in Druid handler
 Key: HIVE-15928
 URL: https://issues.apache.org/jira/browse/HIVE-15928
 Project: Hive
  Issue Type: Sub-task
  Components: Druid integration
Affects Versions: 2.2.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


Even if we split a Select query along its time dimension, parallelization is 
limited as all queries will hit the broker node. Instead, we can interrogate 
the broker to get the Druid nodes that contain the data, and query those nodes 
directly.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15927) HIve does not insert UTF-8 characters properly when insert query is triggered.

2017-02-15 Thread JaySenSharma (JIRA) 
JaySenSharma created HIVE-15927:
---

 Summary: HIve does not insert UTF-8 characters properly when 
insert query is triggered.
 Key: HIVE-15927
 URL: https://issues.apache.org/jira/browse/HIVE-15927
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 1.2.1, 1.2.0
 Environment: All
Reporter: JaySenSharma


- 
If we create a table as following from Ambari Hive View (or using any other 
approach)

{code}
CREATE TABLE IF NOT EXISTS currency2 (id int, name string) ROW FORMAT SERDE 
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' WITH 
SERDEPROPERTIES("serialization.encoding"='UTF-8','store.charset'='UTF-8', 
'retrieve.charset'='UTF-8');

INSERT INTO currency2 VALUES (1, "£€₡");

SELECT * FROM currency2;
{code}


- Output from ambari side is attached as "Result.png"

Here we see that the queries were sent to the "hive" correctly as we see the 
following entries appear in the "hiveserver2.log"
{code}
ssh root@hiveserevr2host
cd /var/log/hive
tail -f hiveserver2.log
{code}

Output of the log shows that hiveserver2 received the proper insert query 
statements from Ambari Hive View.

{code}
2017-01-25 09:25:27,589 INFO  [HiveServer2-Handler-Pool: Thread-60]: 
parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: INSERT INTO 
currency2 VALUES (1, "£€₡")

2017-01-25 09:25:28,337 INFO  [HiveServer2-Background-Pool: Thread-244]: 
exec.Task (TezTask.java:build(316)) - Dag name: INSERT INTO currency2 VALUES 
(1, "£€₡")(Stage-1)

2017-01-25 09:25:28,694 INFO  [HiveServer2-Background-Pool: Thread-244]: 
client.TezClient (TezClient.java:submitDAGSession(577)) - Submitted dag to 
TezSession, sessionName=HIVE-bf038f08-853a-4850-80bc-7a6eaded3c2d, 
applicationId=application_1485333705258_0007, dagName=INSERT INTO c
urrency1 VALUES (1, "£€₡")(Stage-1)
{code}

- Even the beeline fails to show the correct entry:
{code}
[root@erie3 ~]# su - hive

[hive@erie3 root]$ beeline
Beeline version 1.2.1000.2.5.3.0-37 by Apache Hive

beeline> !connect 
jdbc:hive2://erie3.example.com:2181,erie1.example.com:2181,erie4.example.com:2181,erie2.example.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

0: jdbc:hive2://erie3.example.com:2181,erie1.> SELECT * FROM currency2;
+---+-+--+
| currency2.id  | currency2.name  |
+---+-+--+
| 1 | ��� |
+---+-+--+

{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15926) Hive 2.1.1 is not supporting any version of Spark

2017-02-15 Thread Hrishieksh (JIRA) 
Hrishieksh created HIVE-15926:
-

 Summary: Hive 2.1.1 is not supporting any version of Spark 
 Key: HIVE-15926
 URL: https://issues.apache.org/jira/browse/HIVE-15926
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 2.1.1
Reporter: Hrishieksh


I have Hive 2.1.1 and trying to integrate with spark so that i can use spark 
instead of mr as engine.
i opened hive cli and run a sql query ,now 
when using Spark version 1.6.1 and 1.6.3 i am geting error 
Exception in thread "main" java.lang.NoSuchFieldError: SPARK_RPC_SERVER_ADDRESS
at 
org.apache.hive.spark.client.rpc.RpcConfiguration.(RpcConfiguration.java:45)

and whn using spark version 2.0.0 or 2.1.0 i am getting error 
org/apache/spark/JavaSparkListener class not found excpetion.

Now tell me what to do with such problem.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)