How to run pre-build tests

2016-05-10 Thread Svetozar Ivanov 

Hello all,

I'm trying to run pre-build tests for a patch in issue 
https://issues.apache.org/jira/browse/HIVE-11233, unfortunately had 
success only the fist time. Any other attempt had no success, I followed 
everything described on page 
https://cwiki.apache.org/confluence/display/Hive/HowToContribute. Will 
be glad if someone shed some light on that.


Best regards,

Svetozar Ivanov

Re: Review Request 47196: HIVE-13350: Support Alter commands for Rely/NoRely novalidate for PK/FK constraints

2016-05-10 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47196/
---

(Updated May 11, 2016, 3:04 a.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Added -ve test cases.


Repository: hive-git


Description
---

Support Alter commands for Rely/NoRely novalidate for PK/FK constraints


Diffs (updated)
-

  metastore/if/hive_metastore.thrift 2eac836 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
8d4e8f3 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
3a63551 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
68972fb 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fea4fc5 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 07d8119 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
e97d4a9 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 7f4b77e 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 25ccced 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 0204fcd 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 981b961 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 7fcbd6a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0a892e8 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 685ab6a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
23a7f6e 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java 38d8d5a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java c6044b9 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveOperationType.java
 810da48 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/Operation2Privilege.java
 3f138fb 
  ql/src/test/queries/clientnegative/alter_table_constraint_duplicate_pk.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/alter_table_constraint_invalid_fk_col1.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/alter_table_constraint_invalid_fk_col2.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/alter_table_constraint_invalid_fk_tbl1.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/alter_table_constraint_invalid_fk_tbl2.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/alter_table_constraint_invalid_pk_col.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/alter_table_constraint_invalid_pk_tbl.q 
PRE-CREATION 
  ql/src/test/queries/clientpositive/create_with_constraints.q 7aabf74 
  ql/src/test/results/clientnegative/alter_table_constraint_duplicate_pk.q.out 
PRE-CREATION 
  
ql/src/test/results/clientnegative/alter_table_constraint_invalid_fk_col1.q.out 
PRE-CREATION 
  
ql/src/test/results/clientnegative/alter_table_constraint_invalid_fk_col2.q.out 
PRE-CREATION 
  
ql/src/test/results/clientnegative/alter_table_constraint_invalid_fk_tbl1.q.out 
PRE-CREATION 
  
ql/src/test/results/clientnegative/alter_table_constraint_invalid_fk_tbl2.q.out 
PRE-CREATION 
  
ql/src/test/results/clientnegative/alter_table_constraint_invalid_pk_col.q.out 
PRE-CREATION 
  
ql/src/test/results/clientnegative/alter_table_constraint_invalid_pk_tbl.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/create_with_constraints.q.out 34dac30 

Diff: https://reviews.apache.org/r/47196/diff/


Testing
---

Local testing done.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 47040: Monitor changes to FairScheduler.xml file and automatically update / validate jobs submitted to fair-scheduler

2016-05-10 Thread Mohit Sabharwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47040/#review132568
---




ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java (line 501)


"configuration" seems too verbose compared to how it's typically named in 
Hive code. 

conf ?



ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java (line 502)


Looks like you have the same conditional check both here and in the 
function you're calling (validateYarnQueue). In affect, we're doing the same 
check twice. Remove one ?



ql/src/java/org/apache/hadoop/hive/ql/session/YarnFairScheduling.java (line 28)


remove "The YarnFairScheduling class is a "



ql/src/java/org/apache/hadoop/hive/ql/session/YarnFairScheduling.java (line 31)


Comment belongs in the javadoc for configureDefualtSchedulerQueue method.



ql/src/java/org/apache/hadoop/hive/ql/session/YarnFairScheduling.java (line 32)


same, i'd use conf just to keep readable & consistent with hive code.



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (lines 
21 - 22)


most new hive code uses org.slf4j.Logger



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 26)


Instead of wildcard, use actual imports.



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 35)


remove "The FileSystemWatcher "



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 37)


remove empty line



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 40)


remove empty line



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 57)


If it's only for testing, make it public and add @VisibleForTesting 
annotation.



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (lines 
74 - 76)


This seems strange.

Why are we closing the watchService() and creating a new one here?

Can the existing watchService() watch more files?

Add comments explaining why this is necessary.



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 
105)


Please convert this to javadoc. Same in other places.

Comments are typically used inside methods by convention.



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 
107)


do we not need to also close watchService here ?

if (watchService != null) {
   watchService.close()
}



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (lines 
115 - 123)


Does the entire block needs to be synchronized ? Or just 
watchService.take() ?



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 
125)


Should this be a warning ?

Also, add the exception to the log.

LOG.warn("..." + ex, ex)

This catch block also doesn't have the close() call. Need a finally block?



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 
127)


Should this be a warning ?

Also, add the exception to the log.

LOG.warn("..." + ex, ex)



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 
128)


You probably need to add a finally() block and move the this.close() in 
that block.



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 
132)


Why are we setting this flag here ? It's not synchronized either.



shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java (line 
136)


Should this be synchronized at all ? The event is not shared state, right ? 
Are we mutating any shared state in this method ?

Also, make it private ?



shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerQueueAllocator.java

Review Request 47204: HIVE-13549 : Remove jdk version specific out files from Hive2

2016-05-10 Thread Mohit Sabharwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47204/
---

Review request for hive.


Bugs: HIVE-13549
https://issues.apache.org/jira/browse/HIVE-13549


Repository: hive-git


Description
---

HIVE-13549 : Remove jdk version specific out files from Hive2

This patch:
1) Removes the JAVA_VERSION_SPECIFIC_OUTPUT hint from q files
2) Removes the corresponding *java1.7.out and *java1.8.out golden files.
3) Regenerates the corresponding new q.out files

Note that in all cases, the java1.7.out version was more recent than java1.8.out
version, because the latter wwas not kept up to date with the changes.
Note: In some cases, three versions of out file existed (q.out, java1.7.out,
java1.8.out). So, in cases where you see diff in an existing golden file or 
differences between the new q.out file and it's java1.8.out file, it's that
file catching up to the java1.7.out version.

Note that the version specific hint and version specific files
remain for avro_timestamp_win.q. This is a windows specific golden
file. I will need to ping some other community member more familiar
with this test to regenerate.


Diffs
-

  ql/src/test/queries/clientnegative/columnstats_partlvl_invalid_values.q 
712ece7b7d24218e61b909c9eb2025ce0965 
  ql/src/test/queries/clientpositive/authorization_explain.q 
6a9475cfc011e0e2403517b7a67c09129ed19569 
  ql/src/test/queries/clientpositive/avro_date.q 
15c07de0c256a6e317ab5386e90eaf8ee3b9b8fd 
  ql/src/test/queries/clientpositive/avro_deserialize_map_null.q 
962e649d2cd5149aec24d2020bbce986e321cc60 
  ql/src/test/queries/clientpositive/avro_nullable_fields.q 
9ba744178b6dc1a7f2e7a8294f8056a76889edc7 
  ql/src/test/queries/clientpositive/avro_timestamp.q 
7bf0dc8900e917ea5d84e534a66450218273893f 
  ql/src/test/queries/clientpositive/cbo_rp_outer_join_ppr.q 
c497ce99624452797b93080aef6733d99bda48ff 
  ql/src/test/queries/clientpositive/char_udf1.q 
09012b450d3441a73d05f3c38fc6b25ec7f22e6d 
  ql/src/test/queries/clientpositive/input4.q 
83edbe2021803330e5a491160917f4f9df1b6baf 
  ql/src/test/queries/clientpositive/join0.q 
66f2ef355a537e463f4da0128ed6894620f16b04 
  ql/src/test/queries/clientpositive/list_bucket_dml_10.q 
5d3dade61d106023f2f2385643b268da3af0c5cf 
  ql/src/test/queries/clientpositive/list_bucket_dml_11.q 
2d22d6646c11a3bbec455f207babb8edb154ba7d 
  ql/src/test/queries/clientpositive/list_bucket_dml_12.q 
ac063cc194eee468a1b20d9b1b39b1c216ce6f7a 
  ql/src/test/queries/clientpositive/list_bucket_dml_13.q 
d68ca93845385688e095162c3b0d223f2847e812 
  ql/src/test/queries/clientpositive/list_bucket_dml_2.q 
263a002cd4db123f9c33217c61175681f0006b9f 
  ql/src/test/queries/clientpositive/list_bucket_dml_4.q 
86ff342e7e4dc014399780800764f8277e23 
  ql/src/test/queries/clientpositive/list_bucket_dml_5.q 
ace7ba95677db8905687be879262672c594b46ec 
  ql/src/test/queries/clientpositive/list_bucket_dml_6.q 
56847885eac30bde8df99b03dae319a6b4c439ac 
  ql/src/test/queries/clientpositive/list_bucket_dml_8.q 
d90454332218b48f31aaf74299a4e86298c81409 
  ql/src/test/queries/clientpositive/list_bucket_dml_9.q 
620750cef5fe57ebdb59ac180dcfd397056df25a 
  ql/src/test/queries/clientpositive/outer_join_ppr.q 
497a4d1dc3c36bd78eca7c43410383885b756a97 
  ql/src/test/queries/clientpositive/parquet_map_null.q 
61058f319d3189eb5c68032548b4d97d65f133cc 
  ql/src/test/queries/clientpositive/plan_json.q 
aa2b134fe238ef680cdfe80bff3eb26c6627b4bf 
  ql/src/test/queries/clientpositive/stats_list_bucket.q 
51137a81595f7b35f9ea5c3a99f09593bf5d534f 
  ql/src/test/queries/clientpositive/str_to_map.q 
f2993b1780f5e488c009610650c3b4511a6b 
  ql/src/test/queries/clientpositive/subquery_multiinsert.q 
bea2e13f068374bcc8e1dd04b1bb8466dbdf3d98 
  ql/src/test/queries/clientpositive/subquery_notin_having.q 
8b2914d6eb0d753b32a6328f23f511895640a7ab 
  ql/src/test/queries/clientpositive/varchar_udf1.q 
ff40b310dc0e6687f9db369cd452cb1b1860ead2 
  ql/src/test/queries/clientpositive/vector_cast_constant.q 
c50dd8f4c6ba26ec5b18fcc7fb4536d4a2b39f15 
  
ql/src/test/results/clientnegative/columnstats_partlvl_invalid_values.q.java1.7.out
 4ea70e3c9e7e63b7143d5cbf45251d7727d521f8 
  
ql/src/test/results/clientnegative/columnstats_partlvl_invalid_values.q.java1.8.out
 7cae55e445f3b516bff697fa2f47cac1ba109291 
  ql/src/test/results/clientnegative/columnstats_partlvl_invalid_values.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/authorization_explain.q.java1.7.out 
a9ed0495fcecadbddf1fcfb764e916fbb5406662 
  ql/src/test/results/clientpositive/authorization_explain.q.java1.8.out 
b7ec20981a9322cedcfbdc8da5b1bf0946c98bfe 
  ql/src/test/results/clientpositive/authorization_explain.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/avro_date.q.java1.7.out 
501b983a05917502e5f06c2cbed61d4240f4aa8f 
  ql/src/test/results/clientpositive/avro_date.q.java1.

Review Request 47196: HIVE-13350: Support Alter commands for Rely/NoRely novalidate for PK/FK constraints

2016-05-10 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47196/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

Support Alter commands for Rely/NoRely novalidate for PK/FK constraints


Diffs
-

  metastore/if/hive_metastore.thrift 2eac836 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
8d4e8f3 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
3a63551 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
68972fb 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fea4fc5 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 07d8119 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
e97d4a9 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 7f4b77e 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 25ccced 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 0204fcd 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 981b961 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 7fcbd6a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0a892e8 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 685ab6a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
23a7f6e 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java 38d8d5a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java c6044b9 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveOperationType.java
 810da48 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/Operation2Privilege.java
 3f138fb 
  ql/src/test/queries/clientpositive/create_with_constraints.q 7aabf74 
  ql/src/test/results/clientpositive/create_with_constraints.q.out 34dac30 

Diff: https://reviews.apache.org/r/47196/diff/


Testing
---

Local testing done.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 47040: Monitor changes to FairScheduler.xml file and automatically update / validate jobs submitted to fair-scheduler

2016-05-10 Thread Reuben Kuhnert 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47040/
---

(Updated 5 10, 2016, 10:54 오후)


Review request for hive, Lenni Kuff, Mohit Sabharwal, and Sergio Pena.


Changes
---

Updated diff with unit test


Bugs: HIVE-13696
https://issues.apache.org/jira/browse/HIVE-13696


Repository: hive-git


Description
---

Ensure that jobs sent to YARN with impersonation off are correctly routed to 
the proper queue based on fair-scheduler.xml. Monitor this file for changes and 
validate that jobs can only be sent to queues authorized for the user.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 926f6e8 
  ql/src/java/org/apache/hadoop/hive/ql/session/YarnFairScheduling.java 
PRE-CREATION 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
a0015eb 
  shims/common/src/main/java/org/apache/hadoop/fs/FileSystemWatcher.java 
PRE-CREATION 
  shims/common/src/main/java/org/apache/hadoop/hive/shims/SchedulerShim.java 
63803b8 
  shims/scheduler/pom.xml b36c123 
  
shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerQueueAllocator.java
 PRE-CREATION 
  
shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java
 372244d 
  
shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/QueueAllocator.java
 PRE-CREATION 
  
shims/scheduler/src/test/java/org/apache/hadoop/hive/schshim/TestFairScheduler.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47040/diff/


Testing
---


Thanks,

Reuben Kuhnert

[jira] [Created] (HIVE-13732) Security for LlapOutputFormatService

2016-05-10 Thread Jason Dere (JIRA) 
Jason Dere created HIVE-13732:
-

 Summary: Security for LlapOutputFormatService
 Key: HIVE-13732
 URL: https://issues.apache.org/jira/browse/HIVE-13732
 Project: Hive
  Issue Type: Sub-task
  Components: llap
Reporter: Jason Dere






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13731) LLAP: return LLAP token with the splits

2016-05-10 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-13731:
---

 Summary: LLAP: return LLAP token with the splits
 Key: HIVE-13731
 URL: https://issues.apache.org/jira/browse/HIVE-13731
 Project: Hive
  Issue Type: Sub-task
Reporter: Sergey Shelukhin


Need to return the token with the splits, then take it in LLAPIF and making 
sure it's used when talking to LLAP



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13730) hybridgrace_hashjoin_1.q test gets stuck

2016-05-10 Thread Vikram Dixit K (JIRA) 
Vikram Dixit K created HIVE-13730:
-

 Summary: hybridgrace_hashjoin_1.q test gets stuck
 Key: HIVE-13730
 URL: https://issues.apache.org/jira/browse/HIVE-13730
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 2.1.0
Reporter: Vikram Dixit K
Assignee: Wei Zheng
Priority: Critical


I am seeing hybridgrace_hashjoin_1.q getting stuck on master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] Apache Hive 2.0.1 Release Candidate 1

2016-05-10 Thread Lefty Leverenz 
The release notes include HIVE-11233
 but it isn't in the
code, so I removed 2.0.1 from its Fix Version/s.

-- Lefty

On Thu, May 5, 2016 at 10:42 PM, Prasanth J  wrote:

> +1
> - Verified signature and md5 on binary and src
> - Ran some tpcds scale 1 tests on tez with binary
> - Compiled source and ran some unit tests
> - Verified for SNAPSHOT jars
> - Ran Rat checks on some of the modules
>
> Thanks
> Prasanth
>
>
> > On May 3, 2016, at 11:25 PM, Sergey Shelukhin  wrote:
> >
> > Apache Hive 2.0.1 Release Candidate 1 is available here:
> >
> > http://home.apache.org/~sershe/hive-2.0.1-rc1/
> >
> >
> > Maven artifacts are available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehive-1049/
> >
> >
> > Source tag for RC1 (github mirror) is:
> > https://github.com/apache/hive/releases/tag/release-2.0.1-rc1
> >
> >
> > Voting will conclude in 72 hours.
> >
> > Hive PMC Members: Please test and vote.
> >
> > Thanks.
> >
> >
> >
>
>

[jira] [Created] (HIVE-13729) FileSystem$Cache leaks in FileUtils.checkFileAccessWithImpersonation

2016-05-10 Thread Daniel Dai (JIRA) 
Daniel Dai created HIVE-13729:
-

 Summary: FileSystem$Cache leaks in 
FileUtils.checkFileAccessWithImpersonation
 Key: HIVE-13729
 URL: https://issues.apache.org/jira/browse/HIVE-13729
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Reporter: Daniel Dai
Assignee: Daniel Dai


Didn't invoke FileSystem.closeAllForUGI after checkFileAccess. This results 
leak in FileSystem$Cache and eventually OOM for HS2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47155: compute stats in certain cases fails with NPE

2016-05-10 Thread Vikram Dixit Kumaraswamy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47155/#review132474
---


Ship it!




Ship It!

- Vikram Dixit Kumaraswamy


On May 10, 2016, 4:22 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47155/
> ---
> 
> (Updated May 10, 2016, 4:22 a.m.)
> 
> 
> Review request for hive, Gunther Hagleitner and Vikram Dixit Kumaraswamy.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-13621
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 5aedd10 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseUtils.java 
> e0b449b 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java f330564 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  320dc10 
>   ql/src/test/queries/clientpositive/deleteAnalyze.q PRE-CREATION 
>   ql/src/test/results/clientpositive/deleteAnalyze.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/tez/deleteAnalyze.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47155/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>

[jira] [Created] (HIVE-13728) TestHBaseSchemaTool fails on master

2016-05-10 Thread Ashutosh Chauhan (JIRA) 
Ashutosh Chauhan created HIVE-13728:
---

 Summary: TestHBaseSchemaTool fails on master
 Key: HIVE-13728
 URL: https://issues.apache.org/jira/browse/HIVE-13728
 Project: Hive
  Issue Type: Bug
  Components: HBase Metastore, Test
Affects Versions: 2.1.0
Reporter: Ashutosh Chauhan


Presumably because of HIVE-13597



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46020: skip.header.line.count causes values to be skipped when performing insert values

2016-05-10 Thread Vladislav Pavlenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46020/
---

(Updated Май 10, 2016, 3:43 п.п.)


Review request for hive.


Changes
---

Fixed double copy in patch in case when file has a footer or/and header.


Bugs: HIVE-10176
https://issues.apache.org/jira/browse/HIVE-10176


Repository: hive-git


Description
---

When inserting values in to tables with TBLPROPERTIES 
("skip.header.line.count"="1") the first value listed is also skipped1.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java d8ac6ae 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FooterBuffer.java 6a407af 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java bdda89a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 449bef8 
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 
4a05a62 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java dd14124 
  ql/src/test/queries/clientnegative/file_with_header_footer_negative.q 1f63d94 
  ql/src/test/queries/clientpositive/file_with_header_footer.q 8913e54 
  ql/src/test/results/clientnegative/file_with_header_footer_negative.q.out 
1794ae6 
  ql/src/test/results/clientpositive/file_with_header_footer.q.out ca3dadb 
  ql/src/test/results/clientpositive/skiphf_aggr.q.out aeb4b1b 

Diff: https://reviews.apache.org/r/46020/diff/


Testing
---

All the 2 failures are not related:
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery.

UPD: Now I don't create tmp file. Add methow that upload file in hdfs directly.


Thanks,

Vladislav Pavlenko

Review Request 47185: HIVE-13716

2016-05-10 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47185/
---

Review request for hive and Rui Li.


Bugs: HIVE-13716
https://issues.apache.org/jira/browse/HIVE-13716


Repository: hive-git


Description
---

Parallelize setting file permissions for moveFile()


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java 71c9188 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java c4d3bfb 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java c2c6c65 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java f4a9772 
  shims/common/src/main/java/org/apache/hadoop/hive/io/HdfsUtils.java e931156 

Diff: https://reviews.apache.org/r/47185/diff/


Testing
---

Existing regression tests.


Thanks,

Ashutosh Chauhan

[jira] [Created] (HIVE-13727) Getting error Failed rule: 'orderByClause clusterByClause distributeByClause sortByClause limitClause can only be applied to the whole union.' in subquery

2016-05-10 Thread Rajat Khandelwal (JIRA) 
Rajat Khandelwal created HIVE-13727:
---

 Summary: Getting error Failed rule: 'orderByClause clusterByClause 
distributeByClause sortByClause limitClause can only be applied to the whole 
union.' in subquery 
 Key: HIVE-13727
 URL: https://issues.apache.org/jira/browse/HIVE-13727
 Project: Hive
  Issue Type: Bug
Reporter: Rajat Khandelwal


The error comes in the following query:
{noformat}
SELECT *
FROM
  (SELECT *
   FROM srcpart a
   WHERE a.ds = '2008-04-08'
 AND a.hr = '11'
   ORDER BY a.key LIMIT 5
   UNION ALL
   SELECT *
   FROM srcpart b
   WHERE b.ds = '2008-04-08'
 AND b.hr = '14'
   ORDER BY b.key LIMIT 5) subq
ORDER BY KEY LIMIT 5
{noformat}

But not in the following query:

{noformat}
SELECT *
FROM
  (SELECT *
   FROM
 (SELECT *
  FROM srcpart a
  WHERE a.ds = '2008-04-08'
AND a.hr = '11'
  ORDER BY a.key LIMIT 5) pa
   UNION ALL SELECT *
   FROM
 (SELECT *
  FROM srcpart b
  WHERE b.ds = '2008-04-08'
AND b.hr = '14'
  ORDER BY b.key LIMIT 5) pb) subq
ORDER BY KEY LIMIT 5
{noformat}

The queries are logically identical, the query that's rejected has dummy select 
* clauses around the sub-queries. 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46929: HIVE-13676: Tests failing because metastore doesn't come up

2016-05-10 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46929/
---

(Updated May 10, 2016, 2:01 p.m.)


Review request for hive.


Bugs: HIVE-13676
https://issues.apache.org/jira/browse/HIVE-13676


Repository: hive-git


Description
---

In 5-6 test classes, metastore is required to be up for tests to run. The 
metastore is started in setup Phase asynchronously. But there's no logic to 
wait till the metastore comes up. Hence, sometimes tests run even when 
metastore isn't up and fail.


Diffs (updated)
-

  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
 9fa263d53596378276fabc43b7f44409b4b0ccfb 
  
hcatalog/webhcat/java-client/src/test/java/org/apache/hive/hcatalog/api/TestHCatClient.java
 aa9c7d31f2cbe187e84bfddfe411da1f49943819 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMarkPartitionRemote.java
 7576f3923c7ae2e144263445d0a69e5424489567 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
da3da8b209ad9d390fef5fb8c585422e23a9ff0f 
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreGetMetaConf.java
 3f4561cc4b2a737f18e274779d6e748dcd3abad0 
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java
 922a4bf11cbc5693cc5753b8488c5ae3085dd3f4 
  ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveRemote.java 
ee95e160c858a93c43f96775b360f28f8d6a178f 

Diff: https://reviews.apache.org/r/46929/diff/


Testing
---


Thanks,

Rajat Khandelwal

Re: Review Request 46929: HIVE-13676: Tests failing because metastore doesn't come up

2016-05-10 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46929/
---

(Updated May 10, 2016, 1:59 p.m.)


Review request for hive.


Bugs: HIVE-13676
https://issues.apache.org/jira/browse/HIVE-13676


Repository: hive-git


Description
---

In 5-6 test classes, metastore is required to be up for tests to run. The 
metastore is started in setup Phase asynchronously. But there's no logic to 
wait till the metastore comes up. Hence, sometimes tests run even when 
metastore isn't up and fail.


Diffs (updated)
-

  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
 9fa263d53596378276fabc43b7f44409b4b0ccfb 
  
hcatalog/webhcat/java-client/src/test/java/org/apache/hive/hcatalog/api/TestHCatClient.java
 aa9c7d31f2cbe187e84bfddfe411da1f49943819 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMarkPartitionRemote.java
 7576f3923c7ae2e144263445d0a69e5424489567 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
da3da8b209ad9d390fef5fb8c585422e23a9ff0f 
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreGetMetaConf.java
 3f4561cc4b2a737f18e274779d6e748dcd3abad0 
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java
 922a4bf11cbc5693cc5753b8488c5ae3085dd3f4 
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestRecordReaderImpl.java 
fe87794d6df939ec1f061e10162698d36e9836bd 
  ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveRemote.java 
ee95e160c858a93c43f96775b360f28f8d6a178f 

Diff: https://reviews.apache.org/r/46929/diff/


Testing
---


Thanks,

Rajat Khandelwal

Re: Review Request 46929: HIVE-13676: Tests failing because metastore doesn't come up

2016-05-10 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46929/
---

(Updated May 10, 2016, 1:57 p.m.)


Review request for hive.


Bugs: HIVE-13676
https://issues.apache.org/jira/browse/HIVE-13676


Repository: hive-git


Description
---

In 5-6 test classes, metastore is required to be up for tests to run. The 
metastore is started in setup Phase asynchronously. But there's no logic to 
wait till the metastore comes up. Hence, sometimes tests run even when 
metastore isn't up and fail.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/JavaUtils.java 
87bc99e95fe2ac9b726fbe9aa95f1f9db6898dbc 
  common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 
41d150c520590092327bc742c8f5f47c0901781d 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
 9fa263d53596378276fabc43b7f44409b4b0ccfb 
  
hcatalog/webhcat/java-client/src/test/java/org/apache/hive/hcatalog/api/TestHCatClient.java
 aa9c7d31f2cbe187e84bfddfe411da1f49943819 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMarkPartitionRemote.java
 7576f3923c7ae2e144263445d0a69e5424489567 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
 c04670848183a2efe5ddf253002916022e84c885 
  llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java 
006f70fa3d3a15e8fcb6bdc4f38ea98549e1b4e3 
  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapDaemon.java 
ca6d8f84dcd40a37ac14d9c1942f9e8bed6d45bf 
  metastore/if/hive_metastore.thrift c8d78b6fd3c89fb5c6a336510050255175a91687 
  metastore/scripts/upgrade/derby/033-HIVE-12892.derby.sql 
ab004d92ff0aef2fbaba078aa7ad6143e70f3d02 
  metastore/scripts/upgrade/derby/hive-schema-2.1.0.derby.sql 
dc27afcd581204c3d17de055e00c64d746427aed 
  metastore/scripts/upgrade/derby/upgrade-2.0.0-to-2.1.0.derby.sql 
94c686be76aab629a5b6341218d13c9a5c0d43c1 
  metastore/scripts/upgrade/mssql/018-HIVE-12892.mssql.sql 
a13591d7438498c4aab572699d8d5804c86fe171 
  metastore/scripts/upgrade/mssql/hive-schema-2.1.0.mssql.sql 
d9194ff66477aed70d812a2821cd545cb177c519 
  metastore/scripts/upgrade/mssql/upgrade-2.0.0-to-2.1.0.mssql.sql 
c796126e485faf2017d7fedbba836078915ab36f 
  metastore/scripts/upgrade/mysql/033-HIVE-12892.mysql.sql 
783afe80b93f89a26c581ea9863ccaa52312059e 
  metastore/scripts/upgrade/mysql/hive-schema-2.1.0.mysql.sql 
a6b783cb72815f9998a7116b053b4344ded2505e 
  metastore/scripts/upgrade/mysql/upgrade-2.0.0-to-2.1.0.mysql.sql 
c3f83b3326a583d15bdfad710022760800deab70 
  metastore/scripts/upgrade/oracle/033-HIVE-12892.oracle.sql 
d2bd9d50648548f65fc042c6e2e80c56f1bb408f 
  metastore/scripts/upgrade/oracle/hive-schema-2.1.0.oracle.sql 
d003a167bc20a45a353f699b04d4da1d497f22cb 
  metastore/scripts/upgrade/oracle/upgrade-2.0.0-to-2.1.0.oracle.sql 
a226d9a8e8995b1d541d5e47cd49337ca290b0c9 
  metastore/scripts/upgrade/postgres/032-HIVE-12892.postgres.sql 
302c51a599af147a23c7bd503d297d454df91989 
  metastore/scripts/upgrade/postgres/hive-schema-2.1.0.postgres.sql 
43e984cf4b6babbe6dd31a10d1e0019a667f9ed2 
  metastore/scripts/upgrade/postgres/upgrade-2.0.0-to-2.1.0.postgres.sql 
7fc603f973ac9f5c1696bf7c4fe5a3f9fb55998c 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
990be15f8af80fc94b8f46d7a64711cc2bdb55f9 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
2734a1c0b6a7c6a8aa0e3f6aaaff9a72e954b7ab 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 
2d13e77edaefd3f1abe4abc76b154a5688297931 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
3b3e05e140f4b5c7c63eba2eb5d6874cb3d9cafd 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
36a0f96d087d22c7a8d2d721b28473f74f524ff7 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AbortTxnsRequest.java
 PRE-CREATION 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddDynamicPartitions.java
 19bdf100a4fea7002376db1cec2c473623f44e2f 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClearFileMetadataRequest.java
 cfec32eefe51aeb887498057e5c0ce88eb0b132b 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FireEventRequest.java
 44308cc95fe0def251c3d6755c247b7f4f0403ad 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetAllFunctionsResponse.java
 2c297ef9229562b34e9fbca95bcd02821d61a813 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetChangeVersionRequest.java
 a41033118a68ee35e79ebeb4f78f02c3577d5ac0 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetChangeVersionResult.java
 7a2f74c6892d39d073e72d323f3a51f7227d728c 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetFileMetadataByEx