date:20170511

Re: [Announce] New committer: Vineet Garg

2017-05-11 Thread Rajesh Balamohan

Congrats Vineet!

~Rajesh.B

On Fri, May 12, 2017 at 10:04 AM, Lefty Leverenz 
wrote:

> Congratulations Vineet!
>
> -- Lefty
>
>
> On Wed, May 10, 2017 at 2:08 PM, Vineet Garg 
> wrote:
>
> > Thanks all!
> >
> > > On May 10, 2017, at 12:47 AM, Rui Li  wrote:
> > >
> > > Congrats :)
> > >
> > > On Wed, May 10, 2017 at 2:06 PM, Zoltan Haindrich <
> > > zhaindr...@hortonworks.com> wrote:
> > >
> > >> Congratulations!
> > >>
> > >>
> > >> On 10 May 2017 7:57 a.m., Prasanth Jayachandran <
> > >> pjayachand...@hortonworks.com> wrote:
> > >> Congratulations Vineeth!!
> > >>
> > >> Thanks
> > >> Prasanth
> > >>
> > >>
> > >>
> > >> On Tue, May 9, 2017 at 10:52 PM -0700, "Jesus Camacho Rodriguez" <
> > >> jcama...@apache.org> wrote:
> > >>
> > >>
> > >> Congrats Vineet! Well deserved!
> > >>
> > >> --
> > >> Jesús
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On 5/10/17, 6:45 AM, "Peter Vary"  wrote:
> > >>
> > >>> Congratulations Vineet! :)
> > >>>
> > >>> 2017. máj. 9. 22:25 ezt írta ("Ashutosh Chauhan" ):
> > >>>
> >  The Project Management Committee (PMC) for Apache Hive has invited
> > >> Vineet
> >  Garg to become a committer and we are pleased to announce that he
> has
> >  accepted.
> > 
> >  Welcome, Vineet!
> > 
> >  Thanks,
> >  Ashutosh
> > 
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> > > --
> > > Best regards!
> > > Rui Li
> > > Cell: (+86) 13564950210
> >
> >
>

Re: [Announce] New committer: Vineet Garg

2017-05-11 Thread Lefty Leverenz

Congratulations Vineet!

-- Lefty


On Wed, May 10, 2017 at 2:08 PM, Vineet Garg  wrote:

> Thanks all!
>
> > On May 10, 2017, at 12:47 AM, Rui Li  wrote:
> >
> > Congrats :)
> >
> > On Wed, May 10, 2017 at 2:06 PM, Zoltan Haindrich <
> > zhaindr...@hortonworks.com> wrote:
> >
> >> Congratulations!
> >>
> >>
> >> On 10 May 2017 7:57 a.m., Prasanth Jayachandran <
> >> pjayachand...@hortonworks.com> wrote:
> >> Congratulations Vineeth!!
> >>
> >> Thanks
> >> Prasanth
> >>
> >>
> >>
> >> On Tue, May 9, 2017 at 10:52 PM -0700, "Jesus Camacho Rodriguez" <
> >> jcama...@apache.org> wrote:
> >>
> >>
> >> Congrats Vineet! Well deserved!
> >>
> >> --
> >> Jesús
> >>
> >>
> >>
> >>
> >>
> >> On 5/10/17, 6:45 AM, "Peter Vary"  wrote:
> >>
> >>> Congratulations Vineet! :)
> >>>
> >>> 2017. máj. 9. 22:25 ezt írta ("Ashutosh Chauhan" ):
> >>>
>  The Project Management Committee (PMC) for Apache Hive has invited
> >> Vineet
>  Garg to become a committer and we are pleased to announce that he has
>  accepted.
> 
>  Welcome, Vineet!
> 
>  Thanks,
>  Ashutosh
> 
> >>
> >>
> >>
> >>
> >>
> >
> >
> > --
> > Best regards!
> > Rui Li
> > Cell: (+86) 13564950210
>
>

[jira] [Created] (HIVE-16659) Query plan should reflect hive.spark.use.groupby.shuffle

2017-05-11 Thread Rui Li (JIRA)

Rui Li created HIVE-16659:
-

 Summary: Query plan should reflect hive.spark.use.groupby.shuffle
 Key: HIVE-16659
 URL: https://issues.apache.org/jira/browse/HIVE-16659
 Project: Hive
  Issue Type: Bug
Reporter: Rui Li
Assignee: Rui Li


It's useful to show the shuffle type used in the query plan. Currently it shows 
"GROUP" no matter what we set for hive.spark.use.groupby.shuffle.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16658) TestTimestampTZ.java has missed the ASF header

2017-05-11 Thread Saijin Huang (JIRA)

Saijin Huang created HIVE-16658:
---

 Summary: TestTimestampTZ.java has missed the ASF header
 Key: HIVE-16658
 URL: https://issues.apache.org/jira/browse/HIVE-16658
 Project: Hive
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Saijin Huang
Assignee: Saijin Huang
Priority: Minor
 Attachments: HIVE-16658.1.patch





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 58936: HIVE-16143 : Improve msck repair batching

2017-05-11 Thread Lefty Leverenz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58936/#review174756
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3351-3355 (original), 3351-3355 (patched)


Please use newlines (\n) so the generated template file won't put the 
entire description on one line.  For an example, see SPARK_JOB_MAX_TASKS 
several lines above this.

(The template file is discussed in 
https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration#AdminManualConfiguration-hive-site.xmlandhive-default.xml.template.)

Also, "(Not batch wise)" shouldn't have a capital N and needs a period 
after the closing parenthesis.



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3357-3361 (patched)


Looks good, just some minor suggestions:

- please use newlines (\n) so the generated template file won't put the 
entire description on one line
- "if" seems unnecessary and confusing in "or if batch size is reduced to 0"
- "which ever" should be "whichever"
- a comma before "whichever" might be good
- please end the description with a period


- Lefty Leverenz


On May 11, 2017, 12:04 a.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58936/
> ---
> 
> (Updated May 11, 2017, 12:04 a.m.)
> 
> 
> Review request for hive, Aihua Xu, Sergio Pena, and Sahil Takiar.
> 
> 
> Bugs: HIVE-16143
> https://issues.apache.org/jira/browse/HIVE-16143
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16143 : Improve msck repair batching
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> d3ea824c21f2fbf98177cb12a18019416f36a3f9 
>   common/src/java/org/apache/hive/common/util/RetryUtilities.java 
> PRE-CREATION 
>   common/src/test/org/apache/hive/common/util/TestRetryUtilities.java 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/create_like.q 
> 38f384e4c547d3c93d510b89fccfbc2b8e2cba09 
>   itests/hive-blobstore/src/test/results/clientpositive/create_like.q.out 
> 0d362a716291637404a3859fe81068594d82c9e0 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 
> 2ae1eacb68cef6990ae3f2050af0bed7c8e9843f 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 917e565f28b2c9aaea18033ea3b6b20fa41fcd0a 
>   
> ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/msck_repair_0.q 
> 22542331621ca4ce5277c2f46a4264b7540a4d1e 
>   ql/src/test/queries/clientpositive/msck_repair_1.q 
> ea596cbbd2d4c230f2b5afbe379fc1e8836b6fbd 
>   ql/src/test/queries/clientpositive/msck_repair_2.q 
> d8338211e970ebac68a7471ee0960ccf2d51cba3 
>   ql/src/test/queries/clientpositive/msck_repair_3.q 
> fdefca121a2de361dbd19e7ef34fb220e1733ed2 
>   ql/src/test/queries/clientpositive/msck_repair_batchsize.q 
> e56e97ac36a6544f3e20478fdb0e8fa783a857ef 
>   ql/src/test/results/clientpositive/msck_repair_0.q.out 
> 2e0d9dc423071ebbd9a55606f196cf7752e27b1a 
>   ql/src/test/results/clientpositive/msck_repair_1.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_2.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_3.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_batchsize.q.out 
> ba99024163a1f2c59d59e9ed7ea276c154c99d24 
>   ql/src/test/results/clientpositive/repair.q.out 
> c1834640a35500c521a904a115a718c94546df10 
> 
> 
> Diff: https://reviews.apache.org/r/58936/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

[GitHub] hive pull request #183: HIVE-16645 Commands.java has missed the catch statem...

2017-05-11 Thread txhsj

GitHub user txhsj opened a pull request:

https://github.com/apache/hive/pull/183

HIVE-16645 Commands.java has missed the catch statement and has some â¦

In commands.java, the catch statement is missing and the Resultset 
statement is not closed.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/txhsj/hive master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/183.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #183


commit 706766af52c256999c08f5367d1ec0e02f73600e
Author: é»èµé10202350 
Date:   2017-05-12T02:15:23Z

HIVE-16645 Commands.java has missed the catch statement and has some code 
format errors




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-16657) Replace conf variable with checkDatabaseProduct-like in JDBCStorageHandler

2017-05-11 Thread Gunther Hagleitner (JIRA)

Gunther Hagleitner created HIVE-16657:
-

 Summary: Replace conf variable with checkDatabaseProduct-like in 
JDBCStorageHandler
 Key: HIVE-16657
 URL: https://issues.apache.org/jira/browse/HIVE-16657
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16656) CachedStore: make prewarm and background cache update multithreaded

2017-05-11 Thread Vaibhav Gumashta (JIRA)

Vaibhav Gumashta created HIVE-16656:
---

 Summary: CachedStore: make prewarm and background cache update 
multithreaded
 Key: HIVE-16656
 URL: https://issues.apache.org/jira/browse/HIVE-16656
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 3.0.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16655) LLAP: Avoid preempting fragments before they enter the running state

2017-05-11 Thread Siddharth Seth (JIRA)

Siddharth Seth created HIVE-16655:
-

 Summary: LLAP: Avoid preempting fragments before they enter the 
running state
 Key: HIVE-16655
 URL: https://issues.apache.org/jira/browse/HIVE-16655
 Project: Hive
  Issue Type: Bug
Reporter: Siddharth Seth
Assignee: Siddharth Seth


Currently in the AM, fragments may be preempted as soon as they are allocated, 
without knowing whether they will move into the RUNNING state or not. Leads to 
a lot of unnecessary kills.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 58936: HIVE-16143 : Improve msck repair batching

2017-05-11 Thread Sahil Takiar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58936/#review174737
---


Ship it!




Ship It!

- Sahil Takiar


On May 11, 2017, 12:04 a.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58936/
> ---
> 
> (Updated May 11, 2017, 12:04 a.m.)
> 
> 
> Review request for hive, Aihua Xu, Sergio Pena, and Sahil Takiar.
> 
> 
> Bugs: HIVE-16143
> https://issues.apache.org/jira/browse/HIVE-16143
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16143 : Improve msck repair batching
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> d3ea824c21f2fbf98177cb12a18019416f36a3f9 
>   common/src/java/org/apache/hive/common/util/RetryUtilities.java 
> PRE-CREATION 
>   common/src/test/org/apache/hive/common/util/TestRetryUtilities.java 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/create_like.q 
> 38f384e4c547d3c93d510b89fccfbc2b8e2cba09 
>   itests/hive-blobstore/src/test/results/clientpositive/create_like.q.out 
> 0d362a716291637404a3859fe81068594d82c9e0 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 
> 2ae1eacb68cef6990ae3f2050af0bed7c8e9843f 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 917e565f28b2c9aaea18033ea3b6b20fa41fcd0a 
>   
> ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/msck_repair_0.q 
> 22542331621ca4ce5277c2f46a4264b7540a4d1e 
>   ql/src/test/queries/clientpositive/msck_repair_1.q 
> ea596cbbd2d4c230f2b5afbe379fc1e8836b6fbd 
>   ql/src/test/queries/clientpositive/msck_repair_2.q 
> d8338211e970ebac68a7471ee0960ccf2d51cba3 
>   ql/src/test/queries/clientpositive/msck_repair_3.q 
> fdefca121a2de361dbd19e7ef34fb220e1733ed2 
>   ql/src/test/queries/clientpositive/msck_repair_batchsize.q 
> e56e97ac36a6544f3e20478fdb0e8fa783a857ef 
>   ql/src/test/results/clientpositive/msck_repair_0.q.out 
> 2e0d9dc423071ebbd9a55606f196cf7752e27b1a 
>   ql/src/test/results/clientpositive/msck_repair_1.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_2.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_3.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_batchsize.q.out 
> ba99024163a1f2c59d59e9ed7ea276c154c99d24 
>   ql/src/test/results/clientpositive/repair.q.out 
> c1834640a35500c521a904a115a718c94546df10 
> 
> 
> Diff: https://reviews.apache.org/r/58936/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-11 Thread Vaibhav Gumashta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 73e0290 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 88b9faf 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a83e12e 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c22a1db 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
b438479 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
39b1676 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
f6420f5 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 3e3fd20 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 91d8c2a 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/1/


Testing
---


Thanks,

Vaibhav Gumashta

[jira] [Created] (HIVE-16654) Optimize a combination of avg(), sum(), count(distinct) etc

2017-05-11 Thread Pengcheng Xiong (JIRA)

Pengcheng Xiong created HIVE-16654:
--

 Summary: Optimize a combination of avg(), sum(), count(distinct) 
etc
 Key: HIVE-16654
 URL: https://issues.apache.org/jira/browse/HIVE-16654
 Project: Hive
  Issue Type: Bug
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong


an example rewrite for q28 of tpcds is 
{code}
(select LP as B1_LP ,CNT  as B1_CNT,CNTD as B1_CNTD
  from (select sum(xc0) / sum(xc1) as LP, sum(xc1) as CNT, count(1) as CNTD 
from (select sum(ss_list_price) as xc0, count(ss_list_price) as xc1 from 
store_sales  where 
ss_list_price is not null and ss_quantity between 0 and 5
and (ss_list_price between 11 and 11+10 
 or ss_coupon_amt between 460 and 460+1000
 or ss_wholesale_cost between 14 and 14+20)
 group by ss_list_price) ss0) ss1) B1
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16653) Mergejoin should give itself a correct tag

2017-05-11 Thread Pengcheng Xiong (JIRA)

Pengcheng Xiong created HIVE-16653:
--

 Summary: Mergejoin should give itself a correct tag
 Key: HIVE-16653
 URL: https://issues.apache.org/jira/browse/HIVE-16653
 Project: Hive
  Issue Type: Sub-task
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid

2017-05-11 Thread Vihang Karajgaonkar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59070/
---

(Updated May 11, 2017, 8:06 p.m.)


Review request for hive, Ashutosh Chauhan, Naveen Gangam, Sergio Pena, and 
Sahil Takiar.


Changes
---

Addressed Alan's comments on the JIRA.


Bugs: HIVE-16555
https://issues.apache.org/jira/browse/HIVE-16555


Repository: hive-git


Description
---

HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid


Diffs (updated)
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 88b9faf8394a59de39be55b2dd2315db7a8d5ab4 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
 bc00d11e2a1c9fd66b89f1ceca100aaafe43cfed 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
 b95c25ca00751629577e014801c3fb9f1a99bd70 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java
 ef02968e22363d537f58b6054266bf9bc87033ae 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java
 29768c1d660aac937c0cd1fa15fb70b571007d14 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java
 4a46f7537f3ceb16c45010b88786907109fd1090 
  metastore/if/hive_metastore.thrift ca6a0076d1fbee4b0d904c1bafcc056ab739e4c4 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
ca71711e09de962d1cd2ee2ee72b3fcbbac228bc 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
9042cdb265373cd25ee9050fb59f6547f4dfc669 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 
b4a2a926428d529cc88954552eb561041404877d 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
 19151507cae1921a38028582ce00e34cd00585eb 
  metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 
4fb71839471a1a7b8b8ebd9573212c7d40e9f39d 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 
f2a97997a4aaaf0ed07dc094ee8303717e01284d 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 
8ee84af14f87b23956b6bee268c0092e439c17e0 
  metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 
04e63f3a9b858d79bfa4883c36c2ccce69bf55c4 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
cbcfc72ac73ccfe48dd9b57eb9722ae092e7094b 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
53f81188c1cda13f20bdf757391024e0c289d9f9 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
023a2893c3b04681cccf8e7ff21227601f6b 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
a83e12e8f3e3a2f3149e4bbc09524998d0e8928f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 
c22a1db046814bf987de0df33b79e718b8fd6dc6 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
39b1676eb9d3c344a6e66f06f096f3a9fb1931ca 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
f6420f5b99fc49df8675af9718871b6c01bb 
  
metastore/src/model/org/apache/hadoop/hive/metastore/model/MMetastoreDBProperties.java
 PRE-CREATION 
  metastore/src/model/package.jdo 969e19912791b5f5a2b9c5fa4c43800310f5080c 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 3e3fd20de069503fcedacf60fa1df90279af26b2 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 91d8c2af67ae1e49f8d41f16d8eee361b3b2abf9 


Diff: https://reviews.apache.org/r/59070/diff/4/

Changes: https://reviews.apache.org/r/59070/diff/3-4/


Testing
---


Thanks,

Vihang Karajgaonkar

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid

2017-05-11 Thread Vihang Karajgaonkar



> On May 11, 2017, 6:04 p.m., Sahil Takiar wrote:
> > This is more of a nit: but is get_metastore_uuid the best name, could it 
> > just be get_metastore_id. do clients really need to know that its a uuid 
> > vs. just some id

It is actually creating a UUID instead of just another ID (like a integer). I 
think using uuid instead of id in the name specifies the behavior more 
acurately and can be used by the clients accordingly. If we use id I think it 
may be confusing to the clients because they might not know what kind of id is 
it without actually inspecting it.


> On May 11, 2017, 6:04 p.m., Sahil Takiar wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Lines 6979 (patched)
> > 
> >
> > Is logging the exception and throwing it necessary?

I think it will help in debugging since the client side code does not log the 
exception.


> On May 11, 2017, 6:04 p.m., Sahil Takiar wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> > Lines 177 (patched)
> > 
> >
> > is this the correct logging library?

It was an unused import. Removed it.


> On May 11, 2017, 6:04 p.m., Sahil Takiar wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> > Lines 3519 (patched)
> > 
> >
> > what's the point of this method? considering `getMetastoreDbUuid` just 
> > delegates to it.

One of my initial version of the patch has some more meat to it while was 
changed later on. Refactored it out now.


> On May 11, 2017, 6:04 p.m., Sahil Takiar wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> > Lines 3546 (patched)
> > 
> >
> > will a specific exception be thrown if the guid already exists. then we 
> > could just catch that exception.

The actual exception type was some Datanucleus exception if I remember it 
right. Looks like none of the other methods catch such exception specifically 
so kind of used the same logic here as well.


> On May 11, 2017, 6:04 p.m., Sahil Takiar wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> > Lines 3576-3577 (patched)
> > 
> >
> > is this possible? i thought the key name was a unique key

Technically it should not be possible as of this patch but I think it would be 
good to check for this in case of any inadvertent changes/corruption to the 
schema.


- Vihang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59070/#review174698
---


On May 11, 2017, 6:56 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59070/
> ---
> 
> (Updated May 11, 2017, 6:56 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Naveen Gangam, Sergio Pena, and 
> Sahil Takiar.
> 
> 
> Bugs: HIVE-16555
> https://issues.apache.org/jira/browse/HIVE-16555
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  88b9faf8394a59de39be55b2dd2315db7a8d5ab4 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
>  bc00d11e2a1c9fd66b89f1ceca100aaafe43cfed 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
>  b95c25ca00751629577e014801c3fb9f1a99bd70 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java
>  ef02968e22363d537f58b6054266bf9bc87033ae 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java
>  29768c1d660aac937c0cd1fa15fb70b571007d14 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java
>  4a46f7537f3ceb16c45010b88786907109fd1090 
>   metastore/if/hive_metastore.thrift ca6a0076d1fbee4b0d904c1bafcc056ab739e4c4 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
> ca71711e09de962d1cd2ee2ee72b3fcbbac228bc 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
> 9042cdb265373cd25ee9050fb59f6547f4dfc669 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 
> b4a2a926428d529cc88954552eb561041404877d 
>

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid

2017-05-11 Thread Vihang Karajgaonkar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59070/
---

(Updated May 11, 2017, 6:56 p.m.)


Review request for hive, Ashutosh Chauhan, Naveen Gangam, Sergio Pena, and 
Sahil Takiar.


Changes
---

Added Sahil's suggestions and refactored some log messages


Summary (updated)
-

HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid


Bugs: HIVE-16555
https://issues.apache.org/jira/browse/HIVE-16555


Repository: hive-git


Description (updated)
---

HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid


Diffs (updated)
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 88b9faf8394a59de39be55b2dd2315db7a8d5ab4 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
 bc00d11e2a1c9fd66b89f1ceca100aaafe43cfed 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
 b95c25ca00751629577e014801c3fb9f1a99bd70 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java
 ef02968e22363d537f58b6054266bf9bc87033ae 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java
 29768c1d660aac937c0cd1fa15fb70b571007d14 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java
 4a46f7537f3ceb16c45010b88786907109fd1090 
  metastore/if/hive_metastore.thrift ca6a0076d1fbee4b0d904c1bafcc056ab739e4c4 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
ca71711e09de962d1cd2ee2ee72b3fcbbac228bc 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
9042cdb265373cd25ee9050fb59f6547f4dfc669 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 
b4a2a926428d529cc88954552eb561041404877d 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
c21ded144484f83cf59b989eada5506ed8e9ba3b 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
e3725a543ec44ae46f7475156cae270b37b01196 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/MetastoreDBProperty.java
 PRE-CREATION 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
 19151507cae1921a38028582ce00e34cd00585eb 
  metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 
4fb71839471a1a7b8b8ebd9573212c7d40e9f39d 
  metastore/src/gen/thrift/gen-php/metastore/Types.php 
74f0028c4124c729385954537e4fdaf992eb 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 
f2a97997a4aaaf0ed07dc094ee8303717e01284d 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 
8ee84af14f87b23956b6bee268c0092e439c17e0 
  metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
f26cb5b185cf6ae4b79786e6911f1b052822011a 
  metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 
f1aa9a6a9738d8a2d544c15aaa5c1348a6e2ce6c 
  metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 
04e63f3a9b858d79bfa4883c36c2ccce69bf55c4 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
cbcfc72ac73ccfe48dd9b57eb9722ae092e7094b 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
53f81188c1cda13f20bdf757391024e0c289d9f9 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
023a2893c3b04681cccf8e7ff21227601f6b 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
a83e12e8f3e3a2f3149e4bbc09524998d0e8928f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 
c22a1db046814bf987de0df33b79e718b8fd6dc6 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
39b1676eb9d3c344a6e66f06f096f3a9fb1931ca 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
f6420f5b99fc49df8675af9718871b6c01bb 
  
metastore/src/model/org/apache/hadoop/hive/metastore/model/MMetastoreDBProperties.java
 PRE-CREATION 
  metastore/src/model/package.jdo 969e19912791b5f5a2b9c5fa4c43800310f5080c 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 3e3fd20de069503fcedacf60fa1df90279af26b2 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 91d8c2af67ae1e49f8d41f16d8eee361b3b2abf9 


Diff: https://reviews.apache.org/r/59070/diff/3/

Changes: https://reviews.apache.org/r/59070/diff/2-3/


Testing
---


Thanks,

Vihang Karajgaonkar

[jira] [Created] (HIVE-16652) LlapInputFormat: Seeing "output error" WARN message

2017-05-11 Thread Jason Dere (JIRA)

Jason Dere created HIVE-16652:
-

 Summary: LlapInputFormat: Seeing "output error" WARN message
 Key: HIVE-16652
 URL: https://issues.apache.org/jira/browse/HIVE-16652
 Project: Hive
  Issue Type: Sub-task
  Components: llap
Reporter: Jason Dere
Assignee: Jason Dere


Another warning message I'm seeing in the logs for TestJdbcWithMiniLlap after 
adding the line to close the RecordReader in the test:

{noformat}
2017-05-11T11:08:34,511  WARN [IPC Server handler 0 on 54847] ipc.Server: IPC 
Server handler 0 on 54847, call Call#341 Retry#0 heartbeat({  
containerId=container_6830411502416918223_0003_00_00, requestId=2, 
startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, 
taskAttemptId=attempt_6830411502416918223_0003_0_00_00_0, eventCount=2 }), 
rpc version=2, client version=1, methodsFingerPrint=996603002 from 
10.22.8.180:54849: output error
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16651) LlapProtocolClientProxy stack trace when using llap input format

2017-05-11 Thread Jason Dere (JIRA)

Jason Dere created HIVE-16651:
-

 Summary: LlapProtocolClientProxy stack trace when using llap input 
format
 Key: HIVE-16651
 URL: https://issues.apache.org/jira/browse/HIVE-16651
 Project: Hive
  Issue Type: Sub-task
  Components: llap
Reporter: Jason Dere
Assignee: Jason Dere


Seeing this after LlapBaseRecordReader.close():

{noformat}
16/06/28 22:05:32 WARN LlapProtocolClientProxy: RequestManager shutdown with 
error
java.util.concurrent.CancellationException
at java.util.concurrent.FutureTask.report(FutureTask.java:121)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at 
com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:135)
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1170)
at 
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at 
com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at 
com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
at 
com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:91)
at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
at java.util.concurrent.FutureTask.cancel(FutureTask.java:180)
at 
org.apache.hadoop.hive.llap.tez.LlapProtocolClientProxy.serviceStop(LlapProtocolClientProxy.java:131)
at 
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at 
org.apache.hadoop.hive.llap.ext.LlapTaskUmbilicalExternalClient.serviceStop(LlapTaskUmbilicalExternalClient.java:135)
at 
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at 
org.apache.hadoop.service.AbstractService.close(AbstractService.java:250)
at 
org.apache.hadoop.hive.llap.LlapBaseRecordReader.close(LlapBaseRecordReader.java:84)
at 
org.apache.hadoop.hive.llap.LlapRowRecordReader.close(LlapRowRecordReader.java:80)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_uuid

2017-05-11 Thread Sahil Takiar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59070/#review174698
---



This is more of a nit: but is get_metastore_uuid the best name, could it just 
be get_metastore_id. do clients really need to know that its a uuid vs. just 
some id


itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
Lines 26 (patched)


nit: don't use wildcards in imports



metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 6979 (patched)


Is logging the exception and throwing it necessary?



metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
Lines 2542-2543 (patched)


Can't this just be `return client.get...`



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
Lines 177 (patched)


is this the correct logging library?



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
Lines 3519 (patched)


what's the point of this method? considering `getMetastoreDbUuid` just 
delegates to it.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
Lines 3546 (patched)


will a specific exception be thrown if the guid already exists. then we 
could just catch that exception.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
Lines 3576-3577 (patched)


is this possible? i thought the key name was a unique key


- Sahil Takiar


On May 10, 2017, 7:41 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59070/
> ---
> 
> (Updated May 10, 2017, 7:41 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Naveen Gangam, Sergio Pena, and 
> Sahil Takiar.
> 
> 
> Bugs: HIVE-16555
> https://issues.apache.org/jira/browse/HIVE-16555
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16555 : Add a new thrift API call for get_metastore_uuid
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  88b9faf8394a59de39be55b2dd2315db7a8d5ab4 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
>  bc00d11e2a1c9fd66b89f1ceca100aaafe43cfed 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
>  b95c25ca00751629577e014801c3fb9f1a99bd70 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java
>  ef02968e22363d537f58b6054266bf9bc87033ae 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java
>  29768c1d660aac937c0cd1fa15fb70b571007d14 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java
>  4a46f7537f3ceb16c45010b88786907109fd1090 
>   metastore/if/hive_metastore.thrift ca6a0076d1fbee4b0d904c1bafcc056ab739e4c4 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
> ca71711e09de962d1cd2ee2ee72b3fcbbac228bc 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
> 9042cdb265373cd25ee9050fb59f6547f4dfc669 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 
> b4a2a926428d529cc88954552eb561041404877d 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
> c21ded144484f83cf59b989eada5506ed8e9ba3b 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
> e3725a543ec44ae46f7475156cae270b37b01196 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/MetastoreDBProperty.java
>  PRE-CREATION 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
>  19151507cae1921a38028582ce00e34cd00585eb 
>   metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 
> 4fb71839471a1a7b8b8ebd9573212c7d40e9f39d 
>   metastore/src/gen/thrift/gen-php/metastore/Types.php 
> 74f0028c4124c729385954537e4fdaf992eb 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 
> f2a97997a4aaaf0ed07dc094ee8303717e01284d 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 
> 8ee84af14f87b23956b6bee268c0092e439c17e0 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
> f26cb5b185cf6ae4b79786e6911f1b052822011a 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 
>

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_uuid

2017-05-11 Thread Naveen Gangam


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59070/#review174690
---


Ship it!




Ship It!

- Naveen Gangam


On May 10, 2017, 7:41 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59070/
> ---
> 
> (Updated May 10, 2017, 7:41 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Naveen Gangam, Sergio Pena, and 
> Sahil Takiar.
> 
> 
> Bugs: HIVE-16555
> https://issues.apache.org/jira/browse/HIVE-16555
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16555 : Add a new thrift API call for get_metastore_uuid
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  88b9faf8394a59de39be55b2dd2315db7a8d5ab4 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
>  bc00d11e2a1c9fd66b89f1ceca100aaafe43cfed 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
>  b95c25ca00751629577e014801c3fb9f1a99bd70 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java
>  ef02968e22363d537f58b6054266bf9bc87033ae 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java
>  29768c1d660aac937c0cd1fa15fb70b571007d14 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java
>  4a46f7537f3ceb16c45010b88786907109fd1090 
>   metastore/if/hive_metastore.thrift ca6a0076d1fbee4b0d904c1bafcc056ab739e4c4 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
> ca71711e09de962d1cd2ee2ee72b3fcbbac228bc 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
> 9042cdb265373cd25ee9050fb59f6547f4dfc669 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 
> b4a2a926428d529cc88954552eb561041404877d 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
> c21ded144484f83cf59b989eada5506ed8e9ba3b 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
> e3725a543ec44ae46f7475156cae270b37b01196 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/MetastoreDBProperty.java
>  PRE-CREATION 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
>  19151507cae1921a38028582ce00e34cd00585eb 
>   metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 
> 4fb71839471a1a7b8b8ebd9573212c7d40e9f39d 
>   metastore/src/gen/thrift/gen-php/metastore/Types.php 
> 74f0028c4124c729385954537e4fdaf992eb 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 
> f2a97997a4aaaf0ed07dc094ee8303717e01284d 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 
> 8ee84af14f87b23956b6bee268c0092e439c17e0 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
> f26cb5b185cf6ae4b79786e6911f1b052822011a 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 
> f1aa9a6a9738d8a2d544c15aaa5c1348a6e2ce6c 
>   metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 
> 04e63f3a9b858d79bfa4883c36c2ccce69bf55c4 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
> cbcfc72ac73ccfe48dd9b57eb9722ae092e7094b 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
> 53f81188c1cda13f20bdf757391024e0c289d9f9 
>   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
> 023a2893c3b04681cccf8e7ff21227601f6b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> a83e12e8f3e3a2f3149e4bbc09524998d0e8928f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 
> c22a1db046814bf987de0df33b79e718b8fd6dc6 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 39b1676eb9d3c344a6e66f06f096f3a9fb1931ca 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> f6420f5b99fc49df8675af9718871b6c01bb 
>   
> metastore/src/model/org/apache/hadoop/hive/metastore/model/MMetastoreDBProperties.java
>  PRE-CREATION 
>   metastore/src/model/package.jdo 969e19912791b5f5a2b9c5fa4c43800310f5080c 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  3e3fd20de069503fcedacf60fa1df90279af26b2 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  91d8c2af67ae1e49f8d41f16d8eee361b3b2abf9 
> 
> 
> Diff: https://reviews.apache.org/r/59070/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_uuid

2017-05-11 Thread Naveen Gangam



> On May 10, 2017, 6:08 p.m., Naveen Gangam wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> > Lines 3554 (patched)
> > 
> >
> > Is there a chance of this returning large amounts of entries? (as this 
> > table grows over time or due to some abnormality). 
> > Should we consider using a predicate for the query?
> 
> Vihang Karajgaonkar wrote:
> The query is expected to return a Collection of only one entry since 
> there cannot be multiple rows with the property key 'guid' in the table by 
> its design.

Please feel free to drop this issue if it is not an concern.


> On May 10, 2017, 6:08 p.m., Naveen Gangam wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> > Lines 3558 (patched)
> > 
> >
> > Should the UUID be redacted from the log file? Does this pose a 
> > security vulnerability as these logs get distributed?
> 
> Vihang Karajgaonkar wrote:
> As far as I understand there is no security risk in logging the UUID. But 
> I can remove the log if you think that would be better.

I leave the final decision to you. If you are going to redact it, there is 
another log message that logs the UUID, that you might want to consider 
redacting. Close this issue at will.


> On May 10, 2017, 6:08 p.m., Naveen Gangam wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> > Lines 3562 (patched)
> > 
> >
> > I thought the idea is to be able to have multiple rows for "guid", one 
> > of each HMS instances? I do not see the code calling this API for this 
> > feature, but once we have multiple rows of "guid" in the DB, this call will 
> > fail every time rendering this feature unusable. 
> > will be have a check at startup that there arent multiple rows? Feels 
> > its better to fail-fast at startup than to allow HMS to be in this state. 
> > Am I making sense?
> 
> Vihang Karajgaonkar wrote:
> There cannot be multiple rows with guid property key in the table since 
> there is a unique key constraint on that column. Since this is the UUID of 
> database not HMS service, the idea is to have a single GUID even when there 
> are multiple HMS instances as long as they are accessing the same database 
> instance.

yes, I forgot about that. So given that PROPERTY_KEY is a primary key, there 
should never be such scenario no matter what the users attempt to add to their 
HMS metadata. Feel free to drop this issue as well.


- Naveen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59070/#review174514
---


On May 10, 2017, 7:41 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59070/
> ---
> 
> (Updated May 10, 2017, 7:41 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Naveen Gangam, Sergio Pena, and 
> Sahil Takiar.
> 
> 
> Bugs: HIVE-16555
> https://issues.apache.org/jira/browse/HIVE-16555
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16555 : Add a new thrift API call for get_metastore_uuid
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  88b9faf8394a59de39be55b2dd2315db7a8d5ab4 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
>  bc00d11e2a1c9fd66b89f1ceca100aaafe43cfed 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
>  b95c25ca00751629577e014801c3fb9f1a99bd70 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java
>  ef02968e22363d537f58b6054266bf9bc87033ae 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java
>  29768c1d660aac937c0cd1fa15fb70b571007d14 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java
>  4a46f7537f3ceb16c45010b88786907109fd1090 
>   metastore/if/hive_metastore.thrift ca6a0076d1fbee4b0d904c1bafcc056ab739e4c4 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
> ca71711e09de962d1cd2ee2ee72b3fcbbac228bc 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
> 9042cdb265373cd25ee9050fb59f6547f4dfc669 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 
> b4a2a926428d529cc88954552eb561041404877d 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
> c21ded144484f83cf59b989eada5506ed8e9ba3b 
>

[jira] [Created] (HIVE-16650) NumberFormatException in AcidUtils.parseBaseBucketFilename

2017-05-11 Thread JIRA

Kenneth Jørgensen created HIVE-16650:


 Summary: NumberFormatException in AcidUtils.parseBaseBucketFilename
 Key: HIVE-16650
 URL: https://issues.apache.org/jira/browse/HIVE-16650
 Project: Hive
  Issue Type: Bug
  Components: ORC
Affects Versions: 1.2.0
Reporter: Kenneth Jørgensen


While running a pig query we ran into the following error:

```
job_1494464639124_0002  clickIps,clickIpsGrouped,clicks GROUP_BY
Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: 
serious problem
at 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
at 
org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:667)
at 
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:691)
at 
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:526)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1385)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1382)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1635)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1382)
at 
org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
at 
org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
at java.lang.Thread.run(Thread.java:745)
at 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
Caused by: java.lang.RuntimeException: serious problem
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$Context.waitForTasks(OrcInputFormat.java:466)
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:922)
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:947)
at 
org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:157)
at 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
... 18 more
Caused by: java.lang.NumberFormatException: For input string: 
"59027730032546908947033857045786"
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:495)
at java.lang.Integer.parseInt(Integer.java:527)
at 
org.apache.hadoop.hive.ql.io.AcidUtils.parseBaseBucketFilename(AcidUtils.java:145)
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$FileGenerator.run(OrcInputFormat.java:534)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
```

After some investigation we found this file in our S3 directory: 
`59027730032546908947033857045786_00`.

Slightly different version, but the error occurs here: 
https://github.com/apache/hive/blob/release-1.2.0/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L162

It would appear the regex `[0-9]+_[0-9]+` i detecting the first hash part of 
the filenames as all numbers and makes some assumption about it, causing the 
whole thing to error out with a NumberFormatException.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 59137: HIVE-16602: Implement shared scans with Tez

2017-05-11 Thread Ashutosh Chauhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59137/#review174657
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedScanOptimizer.java
Lines 162 (patched)


Comment on we merge filters from previous scan by ORing with filters from 
current scan.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedScanOptimizer.java
Lines 460 (patched)


I am not sure if this function is needed. When we push filters to TS in 
filterExpr of TSDesc, we never remove them from operator tree. And that is by 
design, because only certain formats can use that (like ORC) and even when they 
use there is no contract that they faithfully apply the filter, since they dont 
do per row filter evaluation, only at stripe level.


- Ashutosh Chauhan


On May 10, 2017, 10:20 a.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59137/
> ---
> 
> (Updated May 10, 2017, 10:20 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Gopal V.
> 
> 
> Bugs: HIVE-16602
> https://issues.apache.org/jira/browse/HIVE-16602
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16602: Implement shared scans with Tez
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 99c26ce80ee1790dd2bba215404e552f665c2ee8 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 
> d0fdb52fbca3d5e2ffa6615f64a86b71ccb8b323 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java 
> 85d46f32821c50c275277acf1195fb763ba2d79d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java 
> 3a6baca4338e447e93125e8a8e3785653ec33801 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedScanOptimizer.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java
>  f78bd7cfb2fbba4a94d13fb627602e489410755c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenSparkSkewJoinProcessor.java
>  c9706117c92dfa2a3918b004b9e22479b54aa07f 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezWork.java 
> c87de16071f9c20bc290f37d1e9ee11ad4a4d5a5 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 
> f469cd29fbd529097fef6b20e97135e32fcd80b3 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java 
> c4fb3f300af4eed2b66ae4ee1418b2764abab8c6 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java 
> 8da85d2436d6656a5e928dc0571a358a59cc6021 
>   ql/src/test/queries/clientpositive/perf/query88.q 
> 2be814e9ca2444dc352dca19c69e51afddccbb4b 
>   ql/src/test/results/clientpositive/llap/auto_join0.q.out 
> cba600169adafd3fb23df5e83aeae188e8288994 
>   ql/src/test/results/clientpositive/llap/auto_join30.q.out 
> a26db555cf70f08e9135c33b6f4ab0c0f6bf0ceb 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out 
> b69d0bdbd9a78bfbcbc73e5456ff3969301b6880 
>   ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 
> 964d05824c93e9091316f721debd88b8e96a9b0b 
>   ql/src/test/results/clientpositive/llap/correlationoptimizer2.q.out 
> b628cb15a8d705e2a5e223b169d6f5d17ece8fd9 
>   ql/src/test/results/clientpositive/llap/correlationoptimizer3.q.out 
> d3cfce851d072afe681828e832325350350f84ef 
>   ql/src/test/results/clientpositive/llap/correlationoptimizer6.q.out 
> 4fec2864a81e5e5acf904800f96df75d27f43c05 
>   ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
> 35dde96ea57cd093e82bdcc271098a276d195b86 
>   ql/src/test/results/clientpositive/llap/except_distinct.q.out 
> 4c32ebcb633c4368a0636f953386d46ed4bd93d2 
>   ql/src/test/results/clientpositive/llap/explainuser_1.q.out 
> 584c3b552091556b0226a5e0df9b5892cdda44e8 
>   ql/src/test/results/clientpositive/llap/explainuser_2.q.out 
> 2a27479276ef1399ff8d9f3ceaf4511f2e9f2144 
>   ql/src/test/results/clientpositive/llap/intersect_merge.q.out 
> 62177859725edab78d7a8ad60c46cc52adfde423 
>   ql/src/test/results/clientpositive/llap/join46.q.out 
> 56f6862a8ea522576b2616497d707c557223c732 
>   ql/src/test/results/clientpositive/llap/limit_join_transpose.q.out 
> 61b5c123cb55ba6b62732b0a5b43bb62b2b3351d 
>   ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 
> dd54dd22a66351a32a5b44b3071bdd56c6d0ac41 
>   ql/src/test/results/clientpositive/llap/llap_nullscan.q.out 
> 7d01c695d022e06f8eda089783e03bfb28fd24e7 
>   ql/src/test/results/clientpositive/llap/mapjoin46.q.out 
> 73960ce26c790e8fdc6b93b2827daae609b7a930 
>   ql/src/test/results/clientpositive/llap/mrr.q.out 
> 1437d5dbafe9485d52d88380fc178020c5e32571 
>

[jira] [Created] (HIVE-16649) CREATE TABLE AS SELECT fails if --auxpath is used when CLI is started

2017-05-11 Thread John Todd (JIRA)

John Todd created HIVE-16649:


 Summary: CREATE TABLE AS SELECT fails if --auxpath is used when 
CLI is started
 Key: HIVE-16649
 URL: https://issues.apache.org/jira/browse/HIVE-16649
 Project: Hive
  Issue Type: Bug
  Components: Beeline, CLI
Affects Versions: 2.1.1, 1.2.1
 Environment: Ubuntu, Centos
Reporter: John Todd


If CLI or Beeline are started with jars added by --auxpath, create table as 
select statements fail.

$ touch MANIFEST.MF
$ jar cf empty.jar MANIFEST.MF
$ echo 'X' > /tmp.dummy.txt
$ hive --auxpath empty.jar
hive> create table dummy (value string);
hive> load data local inpath '/tmp/dummy.txt' into table dummy;
hive> create table dummy2 as select * from dummy;
Query ID = john_20170511105539_6818ee81-7c81-446a-bd6b-1f9e550c9640
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.IllegalArgumentException: Can not create a Path from an empty 
string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
at org.apache.hadoop.fs.Path.(Path.java:134)
at 
org.apache.hadoop.mapreduce.JobResourceUploader.uploadFiles(JobResourceUploader.java:129)
at 
org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:95)
at 
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:190)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at 
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
at 
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:431)
at 
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at 
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Job Submission failed with exception 
'java.lang.IllegalArgumentException(Can not create a Path from an empty string)'
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive> 




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16648) Allow select distinct with group by

2017-05-11 Thread Carter Shanklin (JIRA)

Carter Shanklin created HIVE-16648:
--

 Summary: Allow select distinct with group by
 Key: HIVE-16648
 URL: https://issues.apache.org/jira/browse/HIVE-16648
 Project: Hive
  Issue Type: Sub-task
Reporter: Carter Shanklin


Although there are very few legitimate reasons to have both "select distinct" 
and "group by" in the same query, it is still used from time to time and other 
systems support it.

Illustrating the issue:
{code}
hive> create table test (c1 integer);
OK
Time taken: 0.073 seconds
hive> select distinct c1 from test group by c1;
FAILED: SemanticException 1:38 SELECT DISTINCT and GROUP BY can not be in the 
same query. Error encountered near token 'c1'
{code}




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16647) Improve the validation output to make the output to stderr and stdout more consistent

2017-05-11 Thread Aihua Xu (JIRA)

Aihua Xu created HIVE-16647:
---

 Summary: Improve the validation output to make the output to 
stderr and stdout more consistent
 Key: HIVE-16647
 URL: https://issues.apache.org/jira/browse/HIVE-16647
 Project: Hive
  Issue Type: Sub-task
  Components: Metastore
Affects Versions: 2.2.0
Reporter: Aihua Xu
Assignee: Aihua Xu
Priority: Minor


Some output are printed to stderr or stdout inconsistently. Here are some of 
them. Update to make them more consistent.

*  Version table validation
  When the version table is missing, the err msg goes to stderr
  When the version table is not valid, the err msg goes to stdout with a 
message like "Failed in schema version validation: 
*  Metastore/schema table validation
** When the version table contains the wrong version or there are no rows in 
the version table, err msg goes to stderr
** When there diffs between the schema and metastore tables, the err msg goes 
to stdout




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

2017-05-11 Thread Yibing Shi (JIRA)

Yibing Shi created HIVE-16646:
-

 Summary: Alias in transform ... as clause shouldn't be case 
sensitive
 Key: HIVE-16646
 URL: https://issues.apache.org/jira/browse/HIVE-16646
 Project: Hive
  Issue Type: Bug
  Components: hpl/sql
Reporter: Yibing Shi


Create a table like below:
{code:sql}
CREATE TABLE hive_bug(col1 string);
{code}

Run below query in Hive:
{code}
from hive_bug select transform(col1) using '/bin/cat' as ( string);
{code}

The result would be:
{noformat}
0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
'/bin/cat' as ( string);
..
INFO  : OK
+---+--+
|   |
+---+--+
+---+--+
{noformat}

The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16645) Commands.java has missed the catch statement and has some code format errors

2017-05-11 Thread Saijin Huang (JIRA)

Saijin Huang created HIVE-16645:
---

 Summary: Commands.java has missed the catch statement and has some 
code format errors
 Key: HIVE-16645
 URL: https://issues.apache.org/jira/browse/HIVE-16645
 Project: Hive
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Saijin Huang
Assignee: Saijin Huang
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16644) Hook Change Manager for Insert Overwrite

2017-05-11 Thread Sankar Hariappan (JIRA)

Sankar Hariappan created HIVE-16644:
---

 Summary: Hook Change Manager for Insert Overwrite
 Key: HIVE-16644
 URL: https://issues.apache.org/jira/browse/HIVE-16644
 Project: Hive
  Issue Type: Sub-task
  Components: repl
Affects Versions: 2.1.0
Reporter: Sankar Hariappan
Assignee: Sankar Hariappan


For insert overwrite Hive.moveFile is called to replace contents of existing 
partitions. This should trigger move of old files into $CMROOT.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16643) BeeLine tests output should keep the PREHOOK/POSTHOOK Input/Output orderdering

2017-05-11 Thread Peter Vary (JIRA)

Peter Vary created HIVE-16643:
-

 Summary: BeeLine tests output should keep the PREHOOK/POSTHOOK 
Input/Output orderdering
 Key: HIVE-16643
 URL: https://issues.apache.org/jira/browse/HIVE-16643
 Project: Hive
  Issue Type: New Feature
  Components: Testing Infrastructure
Reporter: Peter Vary
Assignee: Peter Vary


The {{PreExecutePrinter}} and the {{PostExecutePrinter}} prints the query input 
and the output list in alphabetical order in {{printEntities}} method.

Our goal is to have the same output from the BeeLine query tests, and the Cli 
query tests. Since the BeeLine tests are using test specific databases to run 
the tests, and only converting the results in the end to remove this specific 
database names from the output, we have to reorder the lists after this 
conversion.

Raw BeeLine output:
{code}
[..]
INFO  : PREHOOK: Output: create_merge_compressed@src_rc_merge_test
INFO  : PREHOOK: Output: database:create_merge_compressed
[..]
{code}

Before patch BeeLine output:
{code}
[..]
PREHOOK: Output: default@src_rc_merge_test
PREHOOK: Output: database:default
[..]
{code}

Expected output:
{code}
[..]
PREHOOK: Output: database:default
PREHOOK: Output: default@src_rc_merge_test
[..]
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16642) New Events created as part of replv2 potentially break replv1

2017-05-11 Thread Sushanth Sowmyan (JIRA)

Sushanth Sowmyan created HIVE-16642:
---

 Summary: New Events created as part of replv2 potentially break 
replv1
 Key: HIVE-16642
 URL: https://issues.apache.org/jira/browse/HIVE-16642
 Project: Hive
  Issue Type: Sub-task
  Components: repl
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan


We have a couple of new events introduced, such as 
{CREATE,DROP}{INDEX,FUNCTION} since the introduction of replv1, but those which 
do not have a replv1 ReplicationTask associated with them.

Thus, for users like Falcon, we potentially wind up throwing a 
IllegalStateException if replv1 based HiveDR is running on a cluster with these 
updated events.

Thus, we should be more graceful when encountering them, returning a 
NoopReplicationTask equivalent that they can make use of, or ignore, for such 
newer events.

In addition, we should add additional test cases so that we track whether or 
not the creation of these events leads to any backward incompatibility we 
introduce. To this end, if any of the events should change so that we introduce 
a backward incompatibility, we should have these tests fail, and alert us to 
that possibility.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [Announce] New committer: Vineet Garg

Re: [Announce] New committer: Vineet Garg

[jira] [Created] (HIVE-16659) Query plan should reflect hive.spark.use.groupby.shuffle

[jira] [Created] (HIVE-16658) TestTimestampTZ.java has missed the ASF header

Re: Review Request 58936: HIVE-16143 : Improve msck repair batching

[GitHub] hive pull request #183: HIVE-16645 Commands.java has missed the catch statem...

[jira] [Created] (HIVE-16657) Replace conf variable with checkDatabaseProduct-like in JDBCStorageHandler

[jira] [Created] (HIVE-16656) CachedStore: make prewarm and background cache update multithreaded

[jira] [Created] (HIVE-16655) LLAP: Avoid preempting fragments before they enter the running state

Re: Review Request 58936: HIVE-16143 : Improve msck repair batching

Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

[jira] [Created] (HIVE-16654) Optimize a combination of avg(), sum(), count(distinct) etc

[jira] [Created] (HIVE-16653) Mergejoin should give itself a correct tag

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_db_uuid

[jira] [Created] (HIVE-16652) LlapInputFormat: Seeing "output error" WARN message

[jira] [Created] (HIVE-16651) LlapProtocolClientProxy stack trace when using llap input format

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_uuid

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_uuid

Re: Review Request 59070: HIVE-16555 : Add a new thrift API call for get_metastore_uuid

[jira] [Created] (HIVE-16650) NumberFormatException in AcidUtils.parseBaseBucketFilename

Re: Review Request 59137: HIVE-16602: Implement shared scans with Tez

[jira] [Created] (HIVE-16649) CREATE TABLE AS SELECT fails if --auxpath is used when CLI is started

[jira] [Created] (HIVE-16648) Allow select distinct with group by

[jira] [Created] (HIVE-16647) Improve the validation output to make the output to stderr and stdout more consistent

[jira] [Created] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[jira] [Created] (HIVE-16645) Commands.java has missed the catch statement and has some code format errors

[jira] [Created] (HIVE-16644) Hook Change Manager for Insert Overwrite

[jira] [Created] (HIVE-16643) BeeLine tests output should keep the PREHOOK/POSTHOOK Input/Output orderdering

[jira] [Created] (HIVE-16642) New Events created as part of replv2 potentially break replv1

31 matches

Site Navigation

Mail list logo

Footer information