Re: Review Request 43274: HIVE-12918 LLAP should never create embedded metastore when localizing functions

2016-02-05 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43274/#review118111
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (line 2538)


this should be removed and changed to false for now


- Sergey Shelukhin


On Feb. 6, 2016, 12:20 a.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43274/
> ---
> 
> (Updated Feb. 6, 2016, 12:20 a.m.)
> 
> 
> Review request for hive and Sushanth Sowmyan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 73e6c21 
>   
> hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HiveClientCache.java
>  0966581 
>   hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/TestPermsGrp.java 
> 41c6b16 
>   
> hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
>  61efc1a 
>   
> hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatOutputFormat.java
>  f716da9 
>   
> hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatPartitionPublish.java
>  5f9379b 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/AbstractTestAuthorizationApiAuthorizer.java
>  f79372d 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
>  b6b0e6e 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java
>  b8cf94f 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreWithEnvironmentContext.java
>  8a37c11 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java
>  569bfd0 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMarkPartition.java
>  57a5e6b 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java
>  e2c860c 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
>  2f3c04b 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
>  6a14982 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreInitListener.java
>  42232fd 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestPartitionNameWhitelistValidation.java
>  61d08cd 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRetryingHMSHandler.java
>  e51ced7 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/StorageBasedMetastoreTestBase.java
>  157ce62 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestAuthorizationPreEventListener.java
>  b8f5dd1 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestClientSideAuthorizationProvider.java
>  2e2da99 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestMetastoreAuthorizationProvider.java
>  3596e30 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestMultiAuthorizationPreEventListener.java
>  299812f 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/FunctionLocalizer.java
>  ce6112f 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
> 0c30262 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java
>  2b05837 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java
>  9c30689 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 75d2519 
>   
> ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java
>  581a919 
>   ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java 5e53604 
> 
> Diff: https://reviews.apache.org/r/43274/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 43274: HIVE-12918 LLAP should never create embedded metastore when localizing functions

2016-02-05 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43274/
---

(Updated Feb. 6, 2016, 12:20 a.m.)


Review request for hive and Sushanth Sowmyan.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 73e6c21 
  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HiveClientCache.java
 0966581 
  hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/TestPermsGrp.java 
41c6b16 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
 61efc1a 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatOutputFormat.java
 f716da9 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatPartitionPublish.java
 5f9379b 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/AbstractTestAuthorizationApiAuthorizer.java
 f79372d 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java
 b6b0e6e 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java
 b8cf94f 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreWithEnvironmentContext.java
 8a37c11 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java
 569bfd0 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMarkPartition.java
 57a5e6b 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java
 e2c860c 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
 2f3c04b 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
 6a14982 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreInitListener.java
 42232fd 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestPartitionNameWhitelistValidation.java
 61d08cd 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestRetryingHMSHandler.java
 e51ced7 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/StorageBasedMetastoreTestBase.java
 157ce62 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestAuthorizationPreEventListener.java
 b8f5dd1 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestClientSideAuthorizationProvider.java
 2e2da99 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestMetastoreAuthorizationProvider.java
 3596e30 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestMultiAuthorizationPreEventListener.java
 299812f 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/FunctionLocalizer.java
 ce6112f 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
0c30262 
  
metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java
 2b05837 
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java
 9c30689 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 75d2519 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java 
581a919 
  ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java 5e53604 

Diff: https://reviews.apache.org/r/43274/diff/


Testing
---


Thanks,

Sergey Shelukhin

Review Request 43274: HIVE-12918 LLAP should never create embedded metastore when localizing functions

2016-02-05 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43274/
---

Review request for hive and Sushanth Sowmyan.


Repository: hive-git


Description
---

see jira


Diffs
-

  hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HCatUtil.java 
3ee30ed 
  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HiveClientCache.java
 0966581 
  hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/TestPermsGrp.java 
41c6b16 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/HCatBaseTest.java
 1dbd024 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
 61efc1a 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatOutputFormat.java
 f716da9 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatPartitionPublish.java
 5f9379b 
  
hcatalog/server-extensions/src/test/java/org/apache/hive/hcatalog/listener/TestNotificationListener.java
 b667454 
  
hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/mutate/UgiMetaStoreClientFactory.java
 328b6c0 
  
hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java
 bde78e4 
  
hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/mutate/StreamingTestUtils.java
 e874877f 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreMetrics.java
 ca21bb6 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java
 071a17e 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/FunctionLocalizer.java
 bc0ad02 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
09a6aea 
  
metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java
 2b05837 
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java
 922a4bf 
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java
 9c30689 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java efb50b2 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java 
581a919 
  ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java 5e53604 
  ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/CompactorTest.java 
a929c95 

Diff: https://reviews.apache.org/r/43274/diff/


Testing
---


Thanks,

Sergey Shelukhin

[jira] [Created] (HIVE-13018) On branch-1 "RuntimeException: Vectorization is not supported for datatype:LIST"

2016-02-05 Thread Matt McCline (JIRA) 
Matt McCline created HIVE-13018:
---

 Summary: On branch-1 "RuntimeException: Vectorization is not 
supported for datatype:LIST"
 Key: HIVE-13018
 URL: https://issues.apache.org/jira/browse/HIVE-13018
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 1.3.0
Reporter: Matt McCline
Assignee: Matt McCline
Priority: Critical


The Schema Evolution backport needs to include the new complex ColumnVector 
types even though we don't actually allow using those types in vectorization.  
Except for an edge case for count(*) where we do vectorize the query with 
complex data types since they are not referenced.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13017) A child process of MR job needs delegation token from AZURE file system

2016-02-05 Thread Takahiko Saito (JIRA) 
Takahiko Saito created HIVE-13017:
-

 Summary: A child process of MR job needs delegation token from 
AZURE file system
 Key: HIVE-13017
 URL: https://issues.apache.org/jira/browse/HIVE-13017
 Project: Hive
  Issue Type: Bug
  Components: Authentication
Affects Versions: 1.2.1
 Environment: Secure 
Reporter: Takahiko Saito
Assignee: Sushanth Sowmyan


The following query fails:
{noformat}
>>>  create temporary table s10k stored as orc as select * from studenttab10k;
>>>  create temporary table v10k as select * from votertab10k;
>>>  select registration 
from s10k s join v10k v 
on (s.name = v.name) join studentparttab30k p 
on (p.name = v.name) 
where s.age < 25 and v.age < 25 and p.age < 25;
ERROR : Execution failed with exit status: 2
ERROR : Obtaining error information
ERROR : 
Task failed!
Task ID:
  Stage-5

Logs:

ERROR : /var/log/hive/hiveServer2.log
Error: Error while processing statement: FAILED: Execution Error, return code 2 
from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask (state=08S01,code=2)
Aborting command set because "force" is false and command failed: "select 
registration 
from s10k s join v10k v 
on (s.name = v.name) join studentparttab30k p 
on (p.name = v.name) 
where s.age < 25 and v.age < 25 and p.age < 25;"
Closing: 0: 
jdbc:hive2://zk2-hs21-h.hdinsight.net:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@hdinsight.net;transportMode=http;httpPath=cliservice
hiveServer2.log shows:
2016-02-02 18:04:34,182 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - 
2016-02-02 18:04:34,199 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - 
2016-02-02 18:04:34,212 INFO  [HiveServer2-HttpHandler-Pool: Thread-55]: 
thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(127)) - Could not 
validate cookie sent, will try to generate a new cookie
2016-02-02 18:04:34,213 INFO  [HiveServer2-Background-Pool: Thread-517]: 
ql.Driver (Driver.java:checkConcurrency(168)) - Concurrency mode is disabled, 
not creating a lock manager
2016-02-02 18:04:34,219 INFO  [HiveServer2-HttpHandler-Pool: Thread-55]: 
thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(352)) - Failed 
to authenticate with http/_HOST kerberos principal, trying with hive/_HOST 
kerberos principal
2016-02-02 18:04:34,219 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - 
2016-02-02 18:04:34,225 INFO  [HiveServer2-Background-Pool: Thread-517]: 
ql.Driver (Driver.java:execute(1390)) - Setting caller context to query id 
hive_20160202180429_76ab-64d6-4c89-88b0-6355cc5acbd0
2016-02-02 18:04:34,226 INFO  [HiveServer2-Background-Pool: Thread-517]: 
ql.Driver (Driver.java:execute(1393)) - Starting 
command(queryId=hive_20160202180429_76ab-64d6-4c89-88b0-6355cc5acbd0): 
select registration
from s10k s join v10k v
on (s.name = v.name) join studentparttab30k p
on (p.name = v.name)
where s.age < 25 and v.age < 25 and p.age < 25
2016-02-02 18:04:34,228 INFO  [HiveServer2-Background-Pool: Thread-517]: 
hooks.ATSHook (ATSHook.java:(90)) - Created ATS Hook
2016-02-02 18:04:34,229 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - 
2016-02-02 18:04:34,237 INFO  [HiveServer2-HttpHandler-Pool: Thread-55]: 
thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(169)) - Cookie added 
for clientUserName hrt_qa
2016-02-02 18:04:34,238 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - 
2016-02-02 18:04:34,239 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - 
2016-02-02 18:04:34,240 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - 
Query ID = hive_20160202180429_76ab-64d6-4c89-88b0-6355cc5acbd0
2016-02-02 18:04:34,242 INFO  [HiveServer2-Background-Pool: Thread-517]: 
ql.Driver (SessionState.java:printInfo(923)) - Query ID = 
hive_20160202180429_76ab-64d6-4c89-88b0-6355cc5acbd0
Total jobs = 1
2016-02-02 18:04:34,243 INFO  [HiveServer2-Background-Pool: Thread-517]: 
ql.Driver (SessionState.java:printInfo(923)) - Total jobs = 1
2016-02-02 18:04:34,245 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - 
2016-02-02 18:04:34,246 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - 
2016-02-02 18:04:34,247 INFO  [HiveServer2-Background-Pool: Thread-517]: 
log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - 
2016-02-02 18:04:34,258 INFO  [HiveServer2-Background-Pool: Thread-517]: 
ql.Driver (Driver.java:launchTask(1718)) - Starting task [Stage-5:MAPREDLOCAL] 
in serial mode
2016-02-02 18:04:34,28

[jira] [Created] (HIVE-13016) ORC FileDump recovery utility fails in Windows

2016-02-05 Thread Prasanth Jayachandran (JIRA) 
Prasanth Jayachandran created HIVE-13016:


 Summary: ORC FileDump recovery utility fails in Windows
 Key: HIVE-13016
 URL: https://issues.apache.org/jira/browse/HIVE-13016
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.3.0, 2.1.0
Reporter: Jason Dere
Assignee: Prasanth Jayachandran


org.apache.hive.hcatalog.streaming.TestStreaming.testFileDumpCorruptDataFiles
org.apache.hive.hcatalog.streaming.TestStreaming.testFileDumpCorruptSideFiles
java.io.IOException: Unable to move 
file:/E:/hive/hcatalog/streaming/target/tmp/junit4129594478393496260/testing3.db/dimensionTable/delta_001_002/bucket_0
 to 
E:/hive/hcatalog/streaming/target/tmp/E:/hive/hcatalog/streaming/target/tmp/junit4129594478393496260/testing3.db/dimensionTable/delta_001_002/bucket_0
at 
org.apache.hadoop.hive.ql.io.orc.FileDump.moveFiles(FileDump.java:546)^M
at 
org.apache.hadoop.hive.ql.io.orc.FileDump.recoverFile(FileDump.java:513)^M
at 
org.apache.hadoop.hive.ql.io.orc.FileDump.recoverFiles(FileDump.java:428)^M
at org.apache.hadoop.hive.ql.io.orc.FileDump.main(FileDump.java:125)^M
at 
org.apache.hive.hcatalog.streaming.TestStreaming.testFileDumpCorruptSideFiles(TestStreaming.java:1523)^M
Note that FileDump appends the full source path to the backup path when trying 
to recover files (see "E:" in the middle of the destination path).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13015) Add dependency convergence for SLF4J

2016-02-05 Thread Prasanth Jayachandran (JIRA) 
Prasanth Jayachandran created HIVE-13015:


 Summary: Add dependency convergence for SLF4J
 Key: HIVE-13015
 URL: https://issues.apache.org/jira/browse/HIVE-13015
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.1.0
Reporter: Prasanth Jayachandran


In some of the recent test runs, we are seeing multiple bindings for SLF4j that 
causes issues with LOG4j2 logger. 

{code}
SLF4J: Found binding in 
[jar:file:/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1454694331819_0001/container_e06_1454694331819_0001_01_02/app/install/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
{code}

We have added explicit exclusions for slf4j-log4j12 but some library is pulling 
it transitively and it's getting packaged with hive libs. Also hive currently 
uses version 1.7.5 for slf4j. We should add dependency convergence for sl4fj 
and also remove packaging of slf4j-log4j12.*.jar 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43143: HIVE-12923 : CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_grouping_sets4.q failure

2016-02-05 Thread Jesús Camacho Rodríguez 


> On Feb. 5, 2016, 6:21 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java,
> >  line 153
> > 
> >
> > This rule is only applied over the Project children. Thus, why is the 
> > parameter for the method hasAggregateChildWithIndicator e.g. leftJoinChild 
> > instead of leftProj?

OK, you still need to check because you might pull up every column from one of 
the sides...


- Jesús


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43143/#review118045
---


On Feb. 3, 2016, 8:54 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43143/
> ---
> 
> (Updated Feb. 3, 2016, 8:54 p.m.)
> 
> 
> Review request for hive, Jesús Camacho Rodríguez and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver 
> groupby_grouping_sets4.q failure
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java
>  f6b2d84 
> 
> Diff: https://reviews.apache.org/r/43143/diff/
> 
> 
> Testing
> ---
> 
> Precommit runs
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 43143: HIVE-12923 : CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_grouping_sets4.q failure

2016-02-05 Thread Jesús Camacho Rodríguez 


> On Feb. 5, 2016, 6:21 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java,
> >  line 153
> > 
> >
> > This rule is only applied over the Project children. Thus, why is the 
> > parameter for the method hasAggregateChildWithIndicator e.g. leftJoinChild 
> > instead of leftProj?
> 
> Jesús Camacho Rodríguez wrote:
> OK, you still need to check because you might pull up every column from 
> one of the sides...

Please Hari, mark it as fixed if you agree.


- Jesús


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43143/#review118045
---


On Feb. 3, 2016, 8:54 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43143/
> ---
> 
> (Updated Feb. 3, 2016, 8:54 p.m.)
> 
> 
> Review request for hive, Jesús Camacho Rodríguez and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver 
> groupby_grouping_sets4.q failure
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java
>  f6b2d84 
> 
> Diff: https://reviews.apache.org/r/43143/diff/
> 
> 
> Testing
> ---
> 
> Precommit runs
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 43143: HIVE-12923 : CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_grouping_sets4.q failure

2016-02-05 Thread Hari Sankar Sivarama Subramaniyan 


> On Feb. 5, 2016, 6:21 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java,
> >  line 153
> > 
> >
> > This rule is only applied over the Project children. Thus, why is the 
> > parameter for the method hasAggregateChildWithIndicator e.g. leftJoinChild 
> > instead of leftProj?
> 
> Jesús Camacho Rodríguez wrote:
> OK, you still need to check because you might pull up every column from 
> one of the sides...
> 
> Jesús Camacho Rodríguez wrote:
> Please Hari, mark it as fixed if you agree.

leftJoinChild is indeed obtained from getProjectChild(). I took the variable 
name/relevant part of the code except hasAggregateChildWithIndicator ()  from 
super.onMatch(). Maybe I should rename the variables/function name to 
hasAggregateDescendantWithIndicator().

Thanks
Hari


- Hari Sankar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43143/#review118045
---


On Feb. 3, 2016, 8:54 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43143/
> ---
> 
> (Updated Feb. 3, 2016, 8:54 p.m.)
> 
> 
> Review request for hive, Jesús Camacho Rodríguez and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver 
> groupby_grouping_sets4.q failure
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java
>  f6b2d84 
> 
> Diff: https://reviews.apache.org/r/43143/diff/
> 
> 
> Testing
> ---
> 
> Precommit runs
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 43143: HIVE-12923 : CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_grouping_sets4.q failure

2016-02-05 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43143/#review118045
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java
 (line 153)


This rule is only applied over the Project children. Thus, why is the 
parameter for the method hasAggregateChildWithIndicator e.g. leftJoinChild 
instead of leftProj?


- Jesús Camacho Rodríguez


On Feb. 3, 2016, 8:54 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43143/
> ---
> 
> (Updated Feb. 3, 2016, 8:54 p.m.)
> 
> 
> Review request for hive, Jesús Camacho Rodríguez and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver 
> groupby_grouping_sets4.q failure
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java
>  f6b2d84 
> 
> Diff: https://reviews.apache.org/r/43143/diff/
> 
> 
> Testing
> ---
> 
> Precommit runs
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

[jira] [Created] (HIVE-13014) RetryingMetaStoreClient is retrying too aggresievley

2016-02-05 Thread Eugene Koifman (JIRA) 
Eugene Koifman created HIVE-13014:
-

 Summary: RetryingMetaStoreClient is retrying too aggresievley
 Key: HIVE-13014
 URL: https://issues.apache.org/jira/browse/HIVE-13014
 Project: Hive
  Issue Type: Bug
  Components: Metastore, Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Priority: Critical


Not all metastore operations are idempotent.  For example, commit_txn() 
consists of 
1. request from client to server
2. server action
3. ack to client

If network connection is broken after (or during) 2 but before 3 happens, 
RetryingMetastoreClient will retry the operation thus causing an attempt to 
commit the same txn twice (sometimes in concurrently)

The 2nd attempt is guaranteed to fail and thus return an error to the caller 
(which doesn't know the operation is being retried), while the first attempt 
has actually succeeded.  Thus the caller thinks commit failed and will likely 
attempt to redo the transactions - not what we want in most cases.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13013) Further Improve concurrency in TxnHandler

2016-02-05 Thread Eugene Koifman (JIRA) 
Eugene Koifman created HIVE-13013:
-

 Summary: Further Improve concurrency in TxnHandler
 Key: HIVE-13013
 URL: https://issues.apache.org/jira/browse/HIVE-13013
 Project: Hive
  Issue Type: Bug
  Components: Metastore, Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman


There are still a few operations in TxnHandler that run at Serializable 
isolation.
Most or all of them can be dropped to READ_COMMITTED now that we have SELECT 
... FOR UPDATE support.  This will reduce number of deadlocks in the DBs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43145: HIVE-12994

2016-02-05 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43145/
---

(Updated Feb. 5, 2016, 5:52 p.m.)


Review request for hive and John Pullokkaran.


Changes
---

Fixes related to windowing.


Bugs: HIVE-12994
https://issues.apache.org/jira/browse/HIVE-12994


Repository: hive-git


Description
---

Implement support for NULLS FIRST/NULLS LAST


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties 
fd2c924380320007245ffd55af5b7809ce4b5201 
  metastore/if/hive_metastore.thrift 9d8c092ef8ab2d081e014fb84242451e69a37ad8 
  metastore/scripts/upgrade/derby/033-HIVE-12994.derby.sql PRE-CREATION 
  metastore/scripts/upgrade/derby/hive-schema-2.1.0.derby.sql 
660a85a4ab464bd69336d99b231edc47a6657850 
  metastore/scripts/upgrade/derby/upgrade-2.0.0-to-2.1.0.derby.sql 
30de00bf034b5bc4470b4f515cd953586fd93658 
  
metastore/src/gen/protobuf/gen-java/org/apache/hadoop/hive/metastore/Metastore.java
 416ae9d68d99110a2d9c2524e4156a43aac18ea6 
  
metastore/src/gen/protobuf/gen-java/org/apache/hadoop/hive/metastore/hbase/HbaseMetastoreProto.java
 3b2d7b5db638dc6b35f968279d6b5129977e6f1d 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
c501ac0eb1ed8ced83a557e4843b2c52d7464602 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
81577b612282f75e09a5a4a3b036fc03b0808ffb 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Order.java
 cc0e2dd027d2e909ea4a39a742b33218c3f8411f 
  metastore/src/gen/thrift/gen-php/metastore/Types.php 
57d1dafcc75f734000f9c8fcd26f54bdcfc2a808 
  metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
77dd9a6fcdaeea31b1d1bc58d370463a1d46 
  metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 
2cf433bff38ca69d8f13350c205e51fd51dab9a1 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
be54b9ca6e7991e08b95892c8f2a2f460f824434 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
b8087287446ef243045a017768008829b5c4eb66 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseUtils.java 
d6d01bdbb879397d1b28675f5dfaa345d9b1066f 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MOrder.java 
1fa82a4fde9d315d4955ee5698796f58ce8a0921 
  
metastore/src/protobuf/org/apache/hadoop/hive/metastore/hbase/hbase_metastore_proto.proto
 466fdf9dc0b0e9d1c58c9b44977896780963e2ac 
  metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestHBaseStore.java 
29d5a64154fdb9ab9586fb919fbeae7106b1a874 
  
metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestHBaseStoreBitVector.java
 b0d76620b94b25c66b8e0857100e6a8c8343 
  
metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestSharedStorageDescriptor.java
 e0d8ce4dede474ad1b752794379342993bf89fa4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
be6ea63fca44bf2bfbdc4368adc26c3d1448561d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 
91b5ca769f88b04c379f38bad3518d36958f1c71 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 
e9cd450276511c36ecbdd0098c083871273f361d 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HybridHashTableContainer.java
 fdc1dfffd977ef3a71fcf56da19a3b2b91538921 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java
 5c2ff9276b575f93a3df0fd2da378b85f7074a22 
  ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 
cfb9abc380880da2cefe6968fd81dc3b974ec1ed 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java
 7bdd11a1e46ac41f16fe26abebe8371fd1e41275 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
75d2519479fcdc00782049958207e11860ed78f4 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractSMBJoinProc.java 
b57dc77c23767f46c27fad0c5761bd2a79489f18 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketingSortingReduceSinkOptimizer.java
 d5f3057a5c070a22786d3151132dc59d38a0ebb4 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java 
c38c6d7f7b8503fed45ad9034b45064bbe3100dd 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedDynPartitionOptimizer.java
 27b04570b0fc2af41aaa41e83198e640ffc17e80 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
02db680c51f7767cd1369bff3b202eaa2367c481 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
 3f2267dc7926243b82d69cc65400b5b536081e53 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ExprNodeConverter.java
 b42e78f796fb91b2bff3443972ad7512b18d68ee 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java
 a0e374c744c880c6980a8abc7667d36c8ccba7e1 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/ReduceSinkDeDuplication.java
 638b91ec9f6a8cdc7af66d96955c40dc

[jira] [Created] (HIVE-13012) NPE from simple nested ANSI Join

2016-02-05 Thread Dave Nicodemus (JIRA) 
Dave Nicodemus created HIVE-13012:
-

 Summary: NPE from simple nested ANSI Join
 Key: HIVE-13012
 URL: https://issues.apache.org/jira/browse/HIVE-13012
 Project: Hive
  Issue Type: Bug
  Components: Parser
Affects Versions: 1.2.1
Reporter: Dave Nicodemus
Assignee: Pengcheng Xiong


Using hive 1.2.1.2.3  Connecting using JDBC, issuing the following query : 

SELECT COUNT(*) 
FROM nation n 
INNER JOIN (customer c
 INNER JOIN orders o ON c.c_custkey = o.o_custkey)
 ON n.n_nationkey = c.c_nationkey;

Generates the NPE and stack below. Fields are integer data type.
NOTE: Similar stack as https://issues.apache.org/jira/browse/HIVE-11433 

Stack

Caused by: java.lang.NullPointerExcEeption: Remote 
java.lang.NullPointerException: null
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.isPresent(SemanticAnalyzer.java:2046)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondPopulateAlias(SemanticAnalyzer.java:2109)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondPopulateAlias(SemanticAnalyzer.java:2185)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondition(SemanticAnalyzer.java:2445)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondition(SemanticAnalyzer.java:2386)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genJoinTree(SemanticAnalyzer.java:8192)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genJoinTree(SemanticAnalyzer.java:8131)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9709)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9636)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10109)
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:329)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10120)
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:211)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:454)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:314)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1164)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1158)
at 
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13011) HIve 0.14 has introduce transaction feature. but when inserting a row my API stuck at " kill command"

2016-02-05 Thread Gaurav Tomar (JIRA) 
Gaurav Tomar created HIVE-13011:
---

 Summary: HIve 0.14 has introduce transaction feature. but when 
inserting a row my API stuck at " kill command"
 Key: HIVE-13011
 URL: https://issues.apache.org/jira/browse/HIVE-13011
 Project: Hive
  Issue Type: New Feature
  Components: API
Affects Versions: 0.14.0
Reporter: Gaurav Tomar


Hive 0.14 has introduce transactions feature. But we need to configure few 
things before any transaction e.g

set hive.support.concurrency = true
set hive.enforce.bucketing = true
set hive.exec.dynamic.partition.mode = nonstrict
set hive.txn.manager =org.apache.hadoop.hive.ql.lockmgr.DbTxnManager
set hive.compactor.initiator.on = true
set hive.compactor.worker.threads  = 1


to create a table with this command :

CREATE TABLE college(clg_id int,clg_name string,clg_loc string) clustered by 
(clg_id) into 5 buckets stored as orc TBLPROPERTIES('transactional'='true');

Now when I am inserting a row into the table using 

INSERT INTO table college 
values(1,'nec','nlr'),(2,'vit','vlr'),(3,'srm','chen'),(4,'lpu','del'),(5,'stanford','uk'),(6,'JNTUA','atp'),(7,'cambridge','us');


My Hive API stuck at "Kill Command" describe below.

Query ID = training_20160205183232_2bb88ba5-187b-4f9e-b93a-47c91b87c79a
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1424080128507_5229, Tracking URL = 
http://hydetamaster:8088/proxy/application_1424080128507_5229/
Kill Command = /opt/installation/hadoop/bin/hadoop job  -kill 
job_1424080128507_5229

Can anybody please help me out with this ??





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 42960: HIVE-11752

2016-02-05 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42960/
---

(Updated Feb. 5, 2016, 12:09 p.m.)


Review request for hive, Ashutosh Chauhan, John Pullokkaran, and Navis Ryu.


Bugs: HIVE-11752
https://issues.apache.org/jira/browse/HIVE-11752


Repository: hive-git


Description
---

Pre-materializing complex CTE queries


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
73e6c216397b0f3e98d39f52eb079c8337de243f 
  itests/src/test/resources/testconfiguration.properties 
fd2c924380320007245ffd55af5b7809ce4b5201 
  ql/src/java/org/apache/hadoop/hive/ql/Context.java 
746456bf39d1a5d700b42d3e0f2c0dcd4001e1ad 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 
7147a9acfe70dbef256fba1507330e2f1d81c182 
  ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java 
9132a212f8bd3600467c4337a0562b1e64bed908 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
be6ea63fca44bf2bfbdc4368adc26c3d1448561d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 
40c89cb9681567a18181514cbffc497e0b2f937e 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 
a5217eb2781dba7f1f35a8a1f87e6784c7ea59ef 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
46bf04c738917e2c3d9f186319fea6f6f2811d3f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
af1ee20a8636523a77b45bf596daddb1330b09d5 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
e2d404b0bd1d3916e498bc4c4076cdf168ef2272 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSemanticAnalyzer.java 
e1e3eb23f13591114153ec5810a0d6c28245085d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezProcContext.java 
ec5ef0eb2be9bde851c70cddbb167a7dc30c78c3 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
ba1945f542b68069584182d10433d0a33a847027 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 
89897d7e5805e05903181f9c202852a9bfdba552 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 
2c6bb217ec880bcc1d4a1cf3c6c891cf779cff52 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java 
cc462bea07e371629a94e809dba910625cc68508 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 
ea506fc13f6a69600e850b4cdcb7a62f613af8b7 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestGenTezWork.java 
be51edc49581f881a4a92b464e0323bf7661b17f 
  ql/src/test/queries/clientpositive/cte_3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/cte_4.q PRE-CREATION 
  ql/src/test/queries/clientpositive/cte_5.q PRE-CREATION 
  ql/src/test/queries/clientpositive/cte_mat_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/cte_mat_2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/cte_mat_3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/cte_mat_4.q PRE-CREATION 
  ql/src/test/queries/clientpositive/cte_mat_5.q PRE-CREATION 
  ql/src/test/results/clientnegative/analyze1.q.out 
589a6ee1bfe9429e5250f5b78202d7aa586b6973 
  ql/src/test/results/clientnegative/dyn_part1.q.out 
62d01fbef8af967474127e2d5f00bd6d2f4858d7 
  ql/src/test/results/clientpositive/alter_view_as_select.q.out 
c89c0dc8602716ad4247208cfaefbab1eabec8e6 
  ql/src/test/results/clientpositive/cte_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cte_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cte_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cte_mat_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cte_mat_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cte_mat_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cte_mat_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cte_mat_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_mat_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_mat_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_mat_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_mat_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/cte_mat_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/tez_union.q.out 
f5cd30fdaf5794d9320a0ff3a7474840e2dc3a92 
  ql/src/test/results/clientpositive/tez/cte_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/tez/cte_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/tez/cte_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/tez/cte_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/tez/cte_5.q.out PRE-CREATION 
  ql/src/test/results/clientp

[jira] [Created] (HIVE-13010) partitions autogenerated predicates broken

2016-02-05 Thread Stanilovsky Evgeny (JIRA) 
Stanilovsky Evgeny created HIVE-13010:
-

 Summary: partitions autogenerated predicates broken
 Key: HIVE-13010
 URL: https://issues.apache.org/jira/browse/HIVE-13010
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Stanilovsky Evgeny
Priority: Trivial


hi, i`m looking for simalar problem but found only:
https://issues.apache.org/jira/browse/HIVE-9630
it`s looks like the same but you can easily repeat it on testing i hope.
I have two simalar requests , the difference in autogenerated data predicates, 
in first case explain show ful lscan.
'''
set hive.optimize.constant.propagation=true;
explain select * from logs.weather_forecasts where dt between
from_unixtime(unix_timestamp() - 3600*24*3, '-MM-dd') and
from_unixtime(unix_timestamp() - 3600*24*1, '-MM-dd') and
provider_id = 100

STAGE PLANS:
5 Stage: Stage-1
6   Map Reduce
7 Map Operator Tree:
8 TableScan
9   alias: weather_forecasts
10  Statistics: Num rows: 36124837607 Data size: 47395787122046 
Basic stats: PARTIAL Column stats: NONE

and

set hive.optimize.constant.propagation=true;
explain select * from logs.redir_log where dt between
'2016-02-02' and
'2016-02-04' and
pid = 100

0   STAGE DEPENDENCIES:
1 Stage-1 is a root stage
2 Stage-0 depends on stages: Stage-1
3   
4   STAGE PLANS:
5 Stage: Stage-1
6   Map Reduce
7 Map Operator Tree:
8 TableScan
9   alias: redir_log
10  Statistics: Num rows: 2798358420 Data size: 5761819991150 
Basic stats: COMPLETE Column stats: NONE




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)