[jira] [Created] (HIVE-12567) Enhance TxnHandler retry logic to handle ORA-08176

[Eugene Koifman (JIRA)]

Eugene Koifman created HIVE-12567:
-

 Summary: Enhance TxnHandler retry logic to handle ORA-08176
 Key: HIVE-12567
 URL: https://issues.apache.org/jira/browse/HIVE-12567
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Eugene Koifman


{noformat}
FAILED: Error in acquiring locks: Error communicating with the metastore
2015-12-01 09:19:32,459 ERROR [HiveServer2-Background-Pool: Thread-55]: 
ql.Driver (SessionState.java:printError(932)) - FAILED: Error in acquiring 
locks: Error communicating with the metastore
org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the 
metastore
at 
org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:132)
at 
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:227)
at 
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:92)
at 
org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:1029)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1226)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1100)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1095)
at 
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
at 
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
at 
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: MetaException(message:Unable to update transaction database 
java.sql.SQLException: ORA-08176: consistent read failure; rollback data not 
available

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1059)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:522)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:210)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:30)
at 
oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:762)
at 
oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:925)
at 
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:)
at 
oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1309)
at 
oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:422)
at 
com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
at 
org.apache.hadoop.hive.metastore.txn.TxnHandler.getLockInfoFromLockId(TxnHandler.java:1951)
at 
org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:1600)
at 
org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:1576)
at 
org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:480)
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5586)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy8.lock(Unknown Source)
at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 

[jira] [Created] (HIVE-12566) Incorrect result returns when using COALESCE in WHERE condition with LEFT JOIN

[Chaoyu Tang (JIRA)]

Chaoyu Tang created HIVE-12566:
--

 Summary: Incorrect result returns when using COALESCE in WHERE 
condition with LEFT JOIN
 Key: HIVE-12566
 URL: https://issues.apache.org/jira/browse/HIVE-12566
 Project: Hive
  Issue Type: Bug
  Components: Query Planning
Affects Versions: 0.13.0
Reporter: Chaoyu Tang
Priority: Critical


The left join query with on/where clause returns incorrect result (more rows 
are returned). See the reproducible sample below.
Left table with data:
{code}
CREATE TABLE ltable (i int, la int, lk1 string, lk2 string) ROW FORMAT 
DELIMITED FIELDS TERMINATED BY ',';
---
1,\N,CD5415192314304,00071
2,\N,CD5415192225530,00071
{code}
Right  table with data:
{code}
CREATE TABLE rtable (ra int, rk1 string, rk2 string) ROW FORMAT DELIMITED 
FIELDS TERMINATED BY ',';
---
1,CD5415192314304,00071
45,CD5415192314304,00072
{code}
Query:
{code}
SELECT * FROM ltable l LEFT OUTER JOIN rtable r on (l.lk1 = r.rk1 AND l.lk2 = 
r.rk2) WHERE COALESCE(l.la,'EMPTY')=COALESCE(r.ra,'EMPTY');
{code}
Result returns:
{code}
1   NULLCD5415192314304 00071   NULLNULLNULL
2   NULLCD5415192225530 00071   NULLNULLNULL
{code}
The correct result should be
{code}
2   NULLCD5415192225530 00071   NULLNULLNULL
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Precommit unstable?

[Sergio Pena]

I see the last tests run in average of 1 hour. Should I still restart
Jenkins?

On Tue, Dec 1, 2015 at 12:11 PM, Thejas Nair  wrote:

> I killed the current 10+hr test run to see if next test would run
> faster. (Discussed with Prasanth, the author of patch being tested).
>
> However,  I doubt if the next one is going to perform any better. If
> we could try restarting the ptest2 system, I think that might be
> better thing to do (instead of waiting several hours to realize that
> the issue has not gone away).
>
>
> On Tue, Dec 1, 2015 at 10:00 AM, Thejas Nair 
> wrote:
> > Would restart of jenkins or the hosts help with this ?
> >
> >
> > On Tue, Dec 1, 2015 at 9:59 AM, Thejas Nair 
> wrote:
> >> The build running right now has been running for almost 10 hours.
> >> I am wondering if there is something wrong with the hosts being used
> >> right now. Some commands seem to be taking longer and I get some
> >> intermittent errors from the jenkins UI.
> >>
> >>
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6187/
> >> This rsync command seems to have taken a long time to finish -
> >>
> >> 2015-12-01 11:42:41,394  INFO LocalCommand.:35 Starting timeout
> >> 1h rsync -vaPe "ssh -i /home/hiveptest/.ssh/hive-ptest-user-key"
> >> --timeout 600 hiveptest@54.159.202.234:
> /home/hiveptest/54.159.202.234-hiveptest-0/logs/
> >>
> /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-6187/succeeded/TestHBaseMinimrCliDriver
> >> 2015-12-01 12:54:52,522  INFO LocalCommand.:35 Starting timeout
> >> 1h rsync -vaPe "ssh -i /home/hiveptest/.ssh/hive-ptest-user-key"
> >> --timeout 600 hiveptest@54.157.22.221:/home/hiveptest/54.157.22.221
> -hiveptest-0/logs/
> >>
> /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-6187/failed/TestMiniTezCliDriver-script_pipe.q-tez_smb_empty.q-mapjoin_decimal.q-and-12-more
> >> 2015-12-01 12:54:54,240  INFO HostExecutor.executeTests:176 Starting
> >> isolated execution on 54.157.22.221
> >>
> >>
> >>
> >>
> >>
> >> On Tue, Dec 1, 2015 at 8:57 AM, Sergio Pena 
> wrote:
> >>> I see HIVE-12541 files were attached 4 times on JIRA while 'Patch
> >>> Available' status was set. I does not say at what time, but Jenkins
> runs a
> >>> job every 5 min to detect which files must be tested for all "Patch
> >>> Available' JIRAS. That's why you probably saw HIVE-12541 in the queue
> many
> >>> times.
> >>>
> >>> I would recommend to set the JIRA to 'Patch Available' when the patch
> is
> >>> reviewed to avoid many patches to be added to the queue.
> >>>
> >>> On Mon, Nov 30, 2015 at 11:09 PM, Prasanth J 
> wrote:
> >>>
>  Hi All
> 
>  I am noticing something weird in precommit queue. I can see
>  HIVE-12541currenly running, but is also waiting next in the queue.
> The same
>  jira ran few more times recently. I cannot see any other patches in
> the
>  queue. Sometime I see bunch of patches in the queue but it will soon
> be
>  removed. But I am seeing HIVE-12541 running quite often. Has anyone
> seen
>  this issue?
> 
>  Thanks
>  Prasanth
>

Re: Review Request 40589: Implement limit push down through union all in CBO

[John Pullokkaran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40589/#review108692
---



ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java (line 1090)


This should be conditionally turned off.
Limit push through Join is protected by "hive.optimize.limitjointranspose". 
Rename this to "hive.optimize.limitpushdown" or something similar.

You should then add this rule to where limit push through join is.


- John Pullokkaran


On Nov. 23, 2015, 5:59 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40589/
> ---
> 
> (Updated Nov. 23, 2015, 5:59 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-11775
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortUnionReduceRule.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java c005b1a 
>   ql/src/test/queries/clientpositive/cbo_SortUnionTransposeRule.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_ppd_boolean.q 059da68 
>   ql/src/test/queries/clientpositive/parquet_ppd_char.q eaddcb4 
>   ql/src/test/queries/clientpositive/parquet_ppd_date.q 41d0d64 
>   ql/src/test/queries/clientpositive/parquet_ppd_decimal.q dfca486 
>   ql/src/test/queries/clientpositive/parquet_ppd_timestamp.q 1b9f6ff 
>   ql/src/test/queries/clientpositive/parquet_ppd_varchar.q 6449c6d 
>   ql/src/test/results/clientpositive/cbo_SortUnionTransposeRule.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/merge4.q.out 031376f 
>   ql/src/test/results/clientpositive/parquet_ppd_boolean.q.out 6e62ee4 
>   ql/src/test/results/clientpositive/parquet_ppd_char.q.out defaa9d 
>   ql/src/test/results/clientpositive/parquet_ppd_date.q.out 55231e9 
>   ql/src/test/results/clientpositive/parquet_ppd_decimal.q.out a30820e 
>   ql/src/test/results/clientpositive/parquet_ppd_timestamp.q.out 3b3e5b7 
>   ql/src/test/results/clientpositive/parquet_ppd_varchar.q.out 5a62e80 
>   ql/src/test/results/clientpositive/spark/temp_table.q.out 718a8a4 
>   ql/src/test/results/clientpositive/temp_table.q.out a9f2bae 
>   ql/src/test/results/clientpositive/tez/temp_table.q.out 200ccdd 
> 
> Diff: https://reviews.apache.org/r/40589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>

Review Request 40876: Default mode for hive.mapred.mode should be strict

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40876/
---

Review request for hive.


Bugs: HIVE-12413
https://issues.apache.org/jira/browse/HIVE-12413


Repository: hive-git


Description
---

Default mode for hive.mapred.mode should be strict


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e984b6e 
  contrib/src/test/queries/clientnegative/invalid_row_sequence.q f7d7e15 
  contrib/src/test/queries/clientpositive/dboutput.q 4c4f5d5 
  contrib/src/test/queries/clientpositive/serde_regex.q 466f9a6 
  contrib/src/test/queries/clientpositive/serde_typedbytes4.q 21d61ee 
  contrib/src/test/queries/clientpositive/udaf_example_avg.q d8e63f2 
  contrib/src/test/queries/clientpositive/udaf_example_group_concat.q 869eac1 
  contrib/src/test/queries/clientpositive/udaf_example_max.q ff2d66b 
  contrib/src/test/queries/clientpositive/udaf_example_max_n.q 4349647 
  contrib/src/test/queries/clientpositive/udaf_example_min.q 765f205 
  contrib/src/test/queries/clientpositive/udaf_example_min_n.q 436e3c1 
  contrib/src/test/queries/clientpositive/udf_row_sequence.q 061f340 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/HCatBaseTest.java
 b87313a 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatExternalDynamicPartitioned.java
 58764a5 
  
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatLoader.java
 28e711a 
  
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatStorerMulti.java
 a4e44d6 
  
itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/ql/security/TestExtendedAcls.java
 d846a63 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/hbase/HBaseIntegrationTests.java
 85fff23 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestMTQueries.java 
95691f6 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestClientSideAuthorizationProvider.java
 9040b1c 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
 5ba9e56 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/authorization/plugin/TestHiveAuthorizerCheckInvocation.java
 4b48e7e 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java
 dabe434 
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 
7bbacd5 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/TestEmbeddedThriftBinaryCLIService.java
 e7be18a 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java db8b7d6 
  ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java 8616eb0 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestOperators.java e18117a 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestUpdateDeleteSemanticAnalyzer.java
 f0f014c 
  ql/src/test/queries/clientcompare/llap_0.q 0a95282 
  ql/src/test/queries/clientnegative/authorization_uri_import.q a6733b5 
  ql/src/test/queries/clientnegative/bucket_mapjoin_mismatch1.q 6bebb89 
  ql/src/test/queries/clientnegative/ctasnullcol.q b03c172 
  ql/src/test/queries/clientnegative/delete_non_acid_table.q ec3d803 
  ql/src/test/queries/clientnegative/dynamic_partitions_with_whitelist.q 
a45d5a5 
  ql/src/test/queries/clientnegative/index_compact_entry_limit.q 5bb889c 
  ql/src/test/queries/clientnegative/index_compact_size_limit.q c6600e6 
  ql/src/test/queries/clientnegative/insert_into6.q a92ee5c 
  ql/src/test/queries/clientnegative/insert_into_with_schema.q 5f27e08 
  ql/src/test/queries/clientnegative/insert_into_with_schema1.q 98b762f 
  ql/src/test/queries/clientnegative/insert_into_with_schema2.q 7520ec8 
  ql/src/test/queries/clientnegative/join_cond_unqual_ambiguous.q c0da913 
  ql/src/test/queries/clientnegative/join_cond_unqual_ambiguous_vc.q 8e21963 
  ql/src/test/queries/clientnegative/join_nonexistent_part.q dea3406 
  ql/src/test/queries/clientnegative/limit_partition.q d593945 
  ql/src/test/queries/clientnegative/ptf_negative_InvalidValueBoundary.q 
ceafa27 
  ql/src/test/queries/clientnegative/serde_regex2.q 395cfec 
  ql/src/test/queries/clientnegative/smb_mapjoin_14.q 4c93542 
  ql/src/test/queries/clientnegative/stats_partialscan_autogether.q e12b450 
  ql/src/test/queries/clientnegative/union22.q 72f3314 
  ql/src/test/queries/clientnegative/update_non_acid_table.q da46141 
  ql/src/test/queries/clientpositive/acid_join.q dca4d7d 
  ql/src/test/queries/clientpositive/acid_vectorization.q 514d3fa 
  ql/src/test/queries/clientpositive/acid_vectorization_partition.q 8dd1e09 
  ql/src/test/queries/clientpositive/acid_vectorization_project.q 2a5f59a 
  ql/src/test/queries/clientpositive/add_part_multiple.q 9248a7f 
  ql/src/test/queries/clientpositive/allcolref_in_udf.q 020975c 
  ql/src/test/queries/clientpositive/alter_char2.q 7fa9fce 
  

Re: Review Request 40589: Implement limit push down through union all in CBO

[John Pullokkaran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40589/#review108698
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortUnionReduceRule.java
 (line 110)


what if the previous limit got pushed past project/join due to 
limitjointranspose rules.

Then this will fire in a loop.

Limit-Union-Proj-Limit


- John Pullokkaran


On Nov. 23, 2015, 5:59 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40589/
> ---
> 
> (Updated Nov. 23, 2015, 5:59 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-11775
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortUnionReduceRule.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java c005b1a 
>   ql/src/test/queries/clientpositive/cbo_SortUnionTransposeRule.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_ppd_boolean.q 059da68 
>   ql/src/test/queries/clientpositive/parquet_ppd_char.q eaddcb4 
>   ql/src/test/queries/clientpositive/parquet_ppd_date.q 41d0d64 
>   ql/src/test/queries/clientpositive/parquet_ppd_decimal.q dfca486 
>   ql/src/test/queries/clientpositive/parquet_ppd_timestamp.q 1b9f6ff 
>   ql/src/test/queries/clientpositive/parquet_ppd_varchar.q 6449c6d 
>   ql/src/test/results/clientpositive/cbo_SortUnionTransposeRule.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/merge4.q.out 031376f 
>   ql/src/test/results/clientpositive/parquet_ppd_boolean.q.out 6e62ee4 
>   ql/src/test/results/clientpositive/parquet_ppd_char.q.out defaa9d 
>   ql/src/test/results/clientpositive/parquet_ppd_date.q.out 55231e9 
>   ql/src/test/results/clientpositive/parquet_ppd_decimal.q.out a30820e 
>   ql/src/test/results/clientpositive/parquet_ppd_timestamp.q.out 3b3e5b7 
>   ql/src/test/results/clientpositive/parquet_ppd_varchar.q.out 5a62e80 
>   ql/src/test/results/clientpositive/spark/temp_table.q.out 718a8a4 
>   ql/src/test/results/clientpositive/temp_table.q.out a9f2bae 
>   ql/src/test/results/clientpositive/tez/temp_table.q.out 200ccdd 
> 
> Diff: https://reviews.apache.org/r/40589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>

Re: Review Request 40833: HIVE-12471 Secure HS2 web UI with SSL

[Jimmy Xiang]

> On Dec. 2, 2015, 6:41 a.m., Mohit Sabharwal wrote:
> > common/src/java/org/apache/hive/http/HttpServer.java, line 280
> > 
> >
> > Nit: b.keyStorePath to determine ssl is not enabled looks bit unclean. 
> > Maybe add a useSsl boolean to the builder ?

Ok.


> On Dec. 2, 2015, 6:41 a.m., Mohit Sabharwal wrote:
> > common/src/java/org/apache/hive/http/HttpServer.java, line 311
> > 
> >
> > Any reason why it shouldn't just default to what's specified in 
> > HiveConf.HIVE_SERVER2_WEBUI_MAX_THREADS (which is 50) ?

It is because this HttpServer can be used for HMS web UI too. I changed the 
logic a little so that we don't hardcode the number here.


> On Dec. 2, 2015, 6:41 a.m., Mohit Sabharwal wrote:
> > common/src/java/org/apache/hive/http/HttpServer.java, line 302
> > 
> >
> > Add back the comment 
> > 
> > /* Set servlet context attribute that can be used in jsp */
> > 
> > It wasn't clear to me this is related to jsp

Added it to setContextAttributes method. You can take a look the jsp file to 
see how it is used. Basically, this is a way to pass information to jsp page.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40833/#review108613
---


On Dec. 1, 2015, 6:50 p.m., Jimmy Xiang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40833/
> ---
> 
> (Updated Dec. 1, 2015, 6:50 p.m.)
> 
> 
> Review request for hive, Lefty Leverenz, Szehon Ho, and Xuefu Zhang.
> 
> 
> Bugs: HIVE-12471
> https://issues.apache.org/jira/browse/HIVE-12471
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch changed the HttpServer construction to use builder which is 
> flexible. It addes 3 configurations for web UI SSL support. By default, it is 
> disabled.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9e805bd 
>   common/src/java/org/apache/hive/http/HttpServer.java 1ff8d7c 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 204eb5a 
> 
> Diff: https://reviews.apache.org/r/40833/diff/
> 
> 
> Testing
> ---
> 
> Tested web UI with and without SSL, both work as expected.
> 
> 
> Thanks,
> 
> Jimmy Xiang
> 
>

[jira] [Created] (HIVE-12568) Use the same logic finding HS2 host name in Spark client [Spark Branch]

[Xuefu Zhang (JIRA)]

Xuefu Zhang created HIVE-12568:
--

 Summary: Use the same logic finding HS2 host name in Spark client 
[Spark Branch]
 Key: HIVE-12568
 URL: https://issues.apache.org/jira/browse/HIVE-12568
 Project: Hive
  Issue Type: Bug
  Components: Spark
Affects Versions: 1.1.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang


Spark client sends a pair of host name and port number to the remote driver so 
that the driver can connects back to HS2 where the user session is. Spark 
client has its own way determining the host name, and pick one network 
interface if the host happens to have multiple network interfaces. This can be 
problematic. For that, there is parameter, hive.spark.client.server.address, 
which user can pick an interface. Unfortunately, this interface isn't exposed.

Instead of exposing this parameter, we can use the same logic as Hive in 
determining the host name. Therefore, the remote driver connecting to HS2 using 
the same network interface as a HS2 client would do.

There might be a case where user may want the remote driver to use a different 
network. This is rare if at all. Thus, for now it should be sufficient to use 
the same network interface.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40628: HIVE-12505: Insert overwrite in same encrypted zone silently fails to remove some existing files

[Chaoyu Tang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40628/
---

(Updated Dec. 2, 2015, 7:49 p.m.)


Review request for hive.


Changes
---

Thanks Aihua for review. I updated the patch based on our offline discussion


Bugs: HIVE-12505
https://issues.apache.org/jira/browse/HIVE-12505


Repository: hive-git


Description
---

With HDFS Trash enabled but its encryption zone lower than Hive data directory, 
insert overwrite command silently fails to trash the existing files during 
overwrite, which could lead to unexpected incorrect results (more rows returned 
than expected if the destination directory could not be cleaned completely). 
This patch changes include:
1. catch trashing failure due to the probable lower Trash encryption zone, and 
retry with force delete
2. if the destination directory could not be cleaned, throw exception instead 
of swallowing it.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java f943781 
  itests/src/test/resources/testconfiguration.properties 0c5140c 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 488d923 
  ql/src/test/queries/clientpositive/encryption_with_trash.q PRE-CREATION 
  ql/src/test/results/clientpositive/encrypted/encryption_with_trash.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/40628/diff/


Testing
---

1. new unit tests passed
2. submitted for precommit test


Thanks,

Chaoyu Tang

Re: Review Request 40833: HIVE-12471 Secure HS2 web UI with SSL

[Jimmy Xiang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40833/
---

(Updated Dec. 2, 2015, 8:09 p.m.)


Review request for hive, Lefty Leverenz, Szehon Ho, and Xuefu Zhang.


Bugs: HIVE-12471
https://issues.apache.org/jira/browse/HIVE-12471


Repository: hive-git


Description
---

This patch changed the HttpServer construction to use builder which is 
flexible. It addes 3 configurations for web UI SSL support. By default, it is 
disabled.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f96071 
  common/src/java/org/apache/hive/http/HttpServer.java 1ff8d7c 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 204eb5a 

Diff: https://reviews.apache.org/r/40833/diff/


Testing
---

Tested web UI with and without SSL, both work as expected.


Thanks,

Jimmy Xiang

[jira] [Created] (HIVE-12571) Push TypeDescription in to the ReaderImpl and RecordReaderImpl

[Owen O'Malley (JIRA)]

Owen O'Malley created HIVE-12571:


 Summary: Push TypeDescription in to the ReaderImpl and 
RecordReaderImpl
 Key: HIVE-12571
 URL: https://issues.apache.org/jira/browse/HIVE-12571
 Project: Hive
  Issue Type: Sub-task
Reporter: Owen O'Malley


We want to use the TypeDescription rather than List because it 
gives us a much better interface.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12569) Excessive console message from SparkClientImpl [Spark Branch]

[Xuefu Zhang (JIRA)]

Xuefu Zhang created HIVE-12569:
--

 Summary: Excessive console message from SparkClientImpl [Spark 
Branch]
 Key: HIVE-12569
 URL: https://issues.apache.org/jira/browse/HIVE-12569
 Project: Hive
  Issue Type: Bug
  Components: Spark
Affects Versions: 2.0.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang


{code}
15/12/02 11:00:46 INFO client.SparkClientImpl: 15/12/02 11:00:46 INFO Client: 
Application report for application_1442517343449_0038 (state: RUNNING)
15/12/02 11:00:47 INFO client.SparkClientImpl: 15/12/02 11:00:47 INFO Client: 
Application report for application_1442517343449_0038 (state: RUNNING)
15/12/02 11:00:48 INFO client.SparkClientImpl: 15/12/02 11:00:48 INFO Client: 
Application report for application_1442517343449_0038 (state: RUNNING)
15/12/02 11:00:49 INFO client.SparkClientImpl: 15/12/02 11:00:49 INFO Client: 
Application report for application_1442517343449_0038 (state: RUNNING)
15/12/02 11:00:50 INFO client.SparkClientImpl: 15/12/02 11:00:50 INFO Client: 
Application report for application_1442517343449_0038 (state: RUNNING)
{code}
I see this using Hive CLI after a spark job is launched and it goes 
non-stopping even if the job is finished.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12570) Incorrect error message Expression not in GROUP BY key thrown instead of Invalid function

[Hari Sankar Sivarama Subramaniyan (JIRA)]

Hari Sankar Sivarama Subramaniyan created HIVE-12570:


 Summary: Incorrect error message Expression not in GROUP BY key 
thrown instead of Invalid function
 Key: HIVE-12570
 URL: https://issues.apache.org/jira/browse/HIVE-12570
 Project: Hive
  Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan


{source}
explain create table avg_salary_by_supervisor3 as select average(key) as 
key_avg from src group by value;
{source}

We get the following stack trace :
{code}
FAILED: SemanticException [Error 10025]: Line 1:57 Expression not in GROUP BY 
key 'key'
ERROR ql.Driver: FAILED: SemanticException [Error 10025]: Line 1:57 Expression 
not in GROUP BY key 'key'
org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:57 Expression not in 
GROUP BY key 'key'
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:10484)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:10432)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3824)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3603)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8862)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8817)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9668)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9561)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10053)
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:345)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10064)
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:222)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:237)
at 
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:237)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:462)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:317)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1227)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1276)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1152)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1140)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
at 
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:778)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}

Instead of the above error message, it be more appropriate to throw the below 
error :
ERROR ql.Driver: FAILED: SemanticException [Error 10011]: Line 1:58 Invalid 
function 'average'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 35868: HIVE-11107

[Hari Sankar Sivarama Subramaniyan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35868/
---

(Updated Dec. 2, 2015, 8:23 p.m.)


Review request for hive, Ashutosh Chauhan and John Pullokkaran.


Repository: hive-git


Description
---

Support for Performance regression test suite with TPCDS


Diffs (updated)
-

  data/files/tpcds-perf/metastore_export/csv/TABLE_PARAMS.txt PRE-CREATION 
  data/files/tpcds-perf/metastore_export/csv/TAB_COL_STATS.txt PRE-CREATION 
  data/files/tpcds-perf/metastore_export/sql/TAB_COL_STATS.sql PRE-CREATION 
  data/scripts/q_perf_test_init.sql PRE-CREATION 
  itests/qtest/pom.xml 8db6fb0 
  ql/src/test/queries/clientpositive/perf/query12.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query13.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query15.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query17.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query18.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query19.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query20.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query22.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query25.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query26.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query27.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query29.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query31.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query34.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query39.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query40.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query42.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query43.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query45.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query46.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query48.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query50.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query52.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query54.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query55.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query64.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query68.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query7.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query70.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query71.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query72.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query73.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query75.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query76.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query79.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query80.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query82.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query84.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query85.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query87.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query90.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query91.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query92.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query93.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query94.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query95.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query96.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query97.q PRE-CREATION 
  ql/src/test/queries/clientpositive/perf/query98.q PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query12.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query13.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query15.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query17.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query18.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query19.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query20.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query22.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query25.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query26.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query27.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query29.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query3.q.out PRE-CREATION 
  

Re: Review Request 38862: HIVE-11985 handle long typenames from Avro schema in metastore

[Thejas Nair]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38862/#review108703
---



ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java (line 786)


the second argument is not being used in the function


- Thejas Nair


On Oct. 21, 2015, 10:54 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38862/
> ---
> 
> (Updated Oct. 21, 2015, 10:54 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Xuefu Zhang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0fcd39b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 12f3f16 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 076791f 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4058606 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java 9f9b5bc 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 3d1ca93 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 3262887 
>   serde/src/java/org/apache/hadoop/hive/serde2/AbstractSerDe.java c5e78c5 
>   serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerDe.java 0e4e4c6 
> 
> Diff: https://reviews.apache.org/r/38862/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

[jira] [Created] (HIVE-12564) Hive query fails with "Caused by: java.lang.RuntimeException: cannot find field _col1 from []"

[Rajesh Balamohan (JIRA)]

Rajesh Balamohan created HIVE-12564:
---

 Summary: Hive query fails with "Caused by: 
java.lang.RuntimeException: cannot find field _col1 from []"
 Key: HIVE-12564
 URL: https://issues.apache.org/jira/browse/HIVE-12564
 Project: Hive
  Issue Type: Bug
Reporter: Rajesh Balamohan


environment: master branch

{noformat}
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 
killedTasks:503, Vertex vertex_1448429572030_2122_4_06 [Reducer 5] 
killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 6, 
vertexId=vertex_1448429572030_2122_4_07, diagnostics=[Vertex received Kill 
while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, 
failedTasks:0 killedTasks:1, Vertex vertex_1448429572030_2122_4_07 [Reducer 6] 
killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to 
VERTEX_FAILURE. failedVertices:1 killedVertices:1
at 
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:367)
 ~[hive-jdbc-2.0.0-SNAPSHOT-standalone.jar:2.0.0-SNAPSHOT]
at 
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:183)
 ~[hive-jdbc-2.0.0-SNAPSHOT-standalone.jar:2.0.0-SNAPSHOT]
at 
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:73)
 ~[hive-jdbc-2.0.0-SNAPSHOT-standalone.jar:2.0.0-SNAPSHOT]
at 
org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:234)
 [hive-jdbc-2.0.0-SNAPSHOT-standalone.jar:2.0.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method) 
~[?:1.8.0_40]
at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_40]
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
 [hadoop-common-2.7.1.2.3.0.0-2557.jar:?]
at 
org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:247) 
[hive-jdbc-2.0.0-SNAPSHOT-standalone.jar:2.0.0-SNAPSHOT]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[?:1.8.0_40]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_40]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[?:1.8.0_40]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[?:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Vertex failed, 
vertexName=Reducer 5, vertexId=vertex_1448429572030_2122_4_06, 
diagnostics=[Task failed, taskId=task_1448429572030_2122_4_06_78, 
diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: 
attempt_1448429572030_2122_4_06_78_0:java.lang.RuntimeException: 
java.lang.RuntimeException: cannot find field _col1 from []
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:348)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:60)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:60)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:35)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: cannot find field _col1 from []
at 
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:442)
at 
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:147)
at 
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:79)
at 
org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:91)
at 
org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:67)
at 

[jira] [Created] (HIVE-12563) NullPointerException with 3-way Tez merge join

[Jason Dere (JIRA)]

Jason Dere created HIVE-12563:
-

 Summary: NullPointerException with 3-way Tez merge join
 Key: HIVE-12563
 URL: https://issues.apache.org/jira/browse/HIVE-12563
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 1.2.0
Reporter: Jason Dere


Issue occurs in Tez merge joins with 3 way join (A join B join C), where A is 
the big table, and A and C end up having 0 rows after table filters.
Was able to repro this issue with using the sample tables in the Hive unit 
tests:
{noformat}
select
  a.key, b.value, c.value
from
  src a,
  src1 b,
  src c
where
  a.key = b.key and a.key = c.key
  and a.value > 'wal_6789'
  and c.value > 'wal_6789'
{noformat}

{noformat}
], TaskAttempt 3 failed, info=[Error: Failure while running 
task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error 
while closing operators: org.apache.hadoop.hive.ql.metadata.HiveException: 
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
Hive Runtime Error while processing row (tag=1) 
{"key":{"reducesinkkey0":"146"},"value":{"_col0":"val_146"}}
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.RuntimeException: Hive Runtime Error while closing 
operators: org.apache.hadoop.hive.ql.metadata.HiveException: 
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
Hive Runtime Error while processing row (tag=1) 
{"key":{"reducesinkkey0":"146"},"value":{"_col0":"val_146"}}
at 
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:310)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
... 13 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row (tag=1) 
{"key":{"reducesinkkey0":"146"},"value":{"_col0":"val_146"}}
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:312)
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:293)
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinFinalLeftData(CommonMergeJoinOperator.java:501)
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.closeOp(CommonMergeJoinOperator.java:416)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
at 
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:287)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
Hive Runtime Error while processing row (tag=1) 
{"key":{"reducesinkkey0":"146"},"value":{"_col0":"val_146"}}
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:439)
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchNextGroup(CommonMergeJoinOperator.java:407)
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:310)
... 19 more
Caused by: java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row (tag=1) 
{"key":{"reducesinkkey0":"146"},"value":{"_col0":"val_146"}}
at 
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:302)
at 

Re: Review Request 40628: HIVE-12505: Insert overwrite in same encrypted zone silently fails to remove some existing files

[Aihua Xu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40628/#review108724
---

Ship it!


Ship It!

- Aihua Xu


On Dec. 2, 2015, 7:50 p.m., Chaoyu Tang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40628/
> ---
> 
> (Updated Dec. 2, 2015, 7:50 p.m.)
> 
> 
> Review request for hive and Aihua Xu.
> 
> 
> Bugs: HIVE-12505
> https://issues.apache.org/jira/browse/HIVE-12505
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> With HDFS Trash enabled but its encryption zone lower than Hive data 
> directory, insert overwrite command silently fails to trash the existing 
> files during overwrite, which could lead to unexpected incorrect results 
> (more rows returned than expected if the destination directory could not be 
> cleaned completely). This patch changes include:
> 1. catch trashing failure due to the probable lower Trash encryption zone, 
> and retry with force delete
> 2. if the destination directory could not be cleaned, throw exception instead 
> of swallowing it.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FileUtils.java f943781 
>   itests/src/test/resources/testconfiguration.properties 0c5140c 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 488d923 
>   ql/src/test/queries/clientpositive/encryption_with_trash.q PRE-CREATION 
>   ql/src/test/results/clientpositive/encrypted/encryption_with_trash.q.out 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/40628/diff/
> 
> 
> Testing
> ---
> 
> 1. new unit tests passed
> 2. submitted for precommit test
> 
> 
> Thanks,
> 
> Chaoyu Tang
> 
>

Re: Review Request 40833: HIVE-12471 Secure HS2 web UI with SSL

[Mohit Sabharwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40833/#review108725
---

Ship it!


The conditional is reverse, otherwise lgtm


common/src/java/org/apache/hive/http/HttpServer.java (line 274)


(!b.useSSL) ?


- Mohit Sabharwal


On Dec. 2, 2015, 8:09 p.m., Jimmy Xiang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40833/
> ---
> 
> (Updated Dec. 2, 2015, 8:09 p.m.)
> 
> 
> Review request for hive, Lefty Leverenz, Szehon Ho, and Xuefu Zhang.
> 
> 
> Bugs: HIVE-12471
> https://issues.apache.org/jira/browse/HIVE-12471
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch changed the HttpServer construction to use builder which is 
> flexible. It addes 3 configurations for web UI SSL support. By default, it is 
> disabled.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f96071 
>   common/src/java/org/apache/hive/http/HttpServer.java 1ff8d7c 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 204eb5a 
> 
> Diff: https://reviews.apache.org/r/40833/diff/
> 
> 
> Testing
> ---
> 
> Tested web UI with and without SSL, both work as expected.
> 
> 
> Thanks,
> 
> Jimmy Xiang
> 
>

[jira] [Created] (HIVE-12577) NPE in LlapTaskCommunicator when unregistering containers

[Siddharth Seth (JIRA)]

Siddharth Seth created HIVE-12577:
-

 Summary: NPE in LlapTaskCommunicator when unregistering containers
 Key: HIVE-12577
 URL: https://issues.apache.org/jira/browse/HIVE-12577
 Project: Hive
  Issue Type: Bug
  Components: llap
Affects Versions: 2.0.0
Reporter: Siddharth Seth
Assignee: Siddharth Seth


{code}
2015-12-02 13:29:00,160 [ERROR] [Dispatcher thread {Central}] 
|common.AsyncDispatcher|: Error in dispatcher thread
java.lang.NullPointerException
at 
org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator$EntityTracker.unregisterContainer(LlapTaskCommunicator.java:586)
at 
org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator.registerContainerEnd(LlapTaskCommunicator.java:188)
at 
org.apache.tez.dag.app.TaskCommunicatorManager.unregisterRunningContainer(TaskCommunicatorManager.java:389)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl.unregisterFromTAListener(AMContainerImpl.java:1121)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtLaunchingTransition.transition(AMContainerImpl.java:699)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtIdleTransition.transition(AMContainerImpl.java:805)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtRunningTransition.transition(AMContainerImpl.java:892)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtRunningTransition.transition(AMContainerImpl.java:887)
at 
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at 
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at 
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at 
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl.handle(AMContainerImpl.java:415)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl.handle(AMContainerImpl.java:72)
at 
org.apache.tez.dag.app.rm.container.AMContainerMap.handle(AMContainerMap.java:60)
at 
org.apache.tez.dag.app.rm.container.AMContainerMap.handle(AMContainerMap.java:36)
at 
org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
at java.lang.Thread.run(Thread.java:745)
2015-12-02 13:29:00,167 [ERROR] [Dispatcher thread {Central}] 
|common.AsyncDispatcher|: Error in dispatcher thread
java.lang.NullPointerException
at 
org.apache.tez.dag.app.TaskCommunicatorManager.unregisterRunningContainer(TaskCommunicatorManager.java:386)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl.unregisterFromTAListener(AMContainerImpl.java:1121)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtLaunchingTransition.transition(AMContainerImpl.java:699)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtIdleTransition.transition(AMContainerImpl.java:805)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtRunningTransition.transition(AMContainerImpl.java:892)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl$StopRequestAtRunningTransition.transition(AMContainerImpl.java:887)
at 
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at 
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at 
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at 
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl.handle(AMContainerImpl.java:415)
at 
org.apache.tez.dag.app.rm.container.AMContainerImpl.handle(AMContainerImpl.java:72)
at 
org.apache.tez.dag.app.rm.container.AMContainerMap.handle(AMContainerMap.java:60)
at 
org.apache.tez.dag.app.rm.container.AMContainerMap.handle(AMContainerMap.java:36)
at 
org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
at java.lang.Thread.run(Thread.java:745)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40833: HIVE-12471 Secure HS2 web UI with SSL

[Jimmy Xiang]

> On Dec. 2, 2015, 9:23 p.m., Mohit Sabharwal wrote:
> > The conditional is reverse, otherwise lgtm

Good catch! Will fix. Thanks.


> On Dec. 2, 2015, 9:23 p.m., Mohit Sabharwal wrote:
> > common/src/java/org/apache/hive/http/HttpServer.java, line 286
> > 
> >
> > (!b.useSSL) ?

Fixed.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40833/#review108725
---


On Dec. 2, 2015, 8:09 p.m., Jimmy Xiang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40833/
> ---
> 
> (Updated Dec. 2, 2015, 8:09 p.m.)
> 
> 
> Review request for hive, Lefty Leverenz, Szehon Ho, and Xuefu Zhang.
> 
> 
> Bugs: HIVE-12471
> https://issues.apache.org/jira/browse/HIVE-12471
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch changed the HttpServer construction to use builder which is 
> flexible. It addes 3 configurations for web UI SSL support. By default, it is 
> disabled.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7f96071 
>   common/src/java/org/apache/hive/http/HttpServer.java 1ff8d7c 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 204eb5a 
> 
> Diff: https://reviews.apache.org/r/40833/diff/
> 
> 
> Testing
> ---
> 
> Tested web UI with and without SSL, both work as expected.
> 
> 
> Thanks,
> 
> Jimmy Xiang
> 
>

[jira] [Created] (HIVE-12572) select partitioned acid table order by throws java.io.FileNotFoundException

[Takahiko Saito (JIRA)]

Takahiko Saito created HIVE-12572:
-

 Summary: select partitioned acid table order by throws 
java.io.FileNotFoundException
 Key: HIVE-12572
 URL: https://issues.apache.org/jira/browse/HIVE-12572
 Project: Hive
  Issue Type: Bug
  Components: Transactions
Affects Versions: 2.0.0
Reporter: Takahiko Saito
Assignee: Alan Gates


Run the below queries:
{noformat}
create table test_acid (a int) partitioned by (b int) clustered by (a) into 2 
buckets stored as orc tblproperties ('transactional'='true');
insert into table test_acid partition (b=1) values (1), (2), (3), (4);
select * from acid_partitioned order by a;
{noformat}

The above fails with the following error:
{noformat}
15/12/02 21:12:30 INFO SessionState: Map 1: 0(+0,-4)/1  Reducer 2: 0/1
Status: Failed
15/12/02 21:12:30 ERROR SessionState: Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1449077191499_0023_1_00, 
diagnostics=[Task failed, taskId=task_1449077191499_0023_1_00_00, 
diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: 
attempt_1449077191499_0023_1_00_00_0:java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: 
java.io.FileNotFoundException: Path is not a file: 
/apps/hive/warehouse/test_acid/b=1
at 
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
at 
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1828)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1712)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:652)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:365)
at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:348)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:60)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:60)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:35)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
java.io.IOException: java.io.FileNotFoundException: Path is not a file: 
/apps/hive/warehouse/test_acid/b=1
at 
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
at 
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1828)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1712)
at 

[jira] [Created] (HIVE-12573) some LLAP (and other) tests are broken

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-12573:
---

 Summary: some LLAP (and other) tests are broken
 Key: HIVE-12573
 URL: https://issues.apache.org/jira/browse/HIVE-12573
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin


It looks like LLAP out files were not updated in some DPP JIRA because the test 
was entirely broken in HiveQA at the time. Some other tests might be broken for 
a simple reason too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12575) Similar to HIVE-12574, collect_set and count() give incorrect result when partition size is smaller than window size

[Aihua Xu (JIRA)]

Aihua Xu created HIVE-12575:
---

 Summary: Similar to HIVE-12574, collect_set and count() give 
incorrect result when partition size is smaller than window size
 Key: HIVE-12575
 URL: https://issues.apache.org/jira/browse/HIVE-12575
 Project: Hive
  Issue Type: Sub-task
  Components: PTF-Windowing
Affects Versions: 2.0.0
Reporter: Aihua Xu
Assignee: Aihua Xu






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12574) windowing function returns incorrect result when the window size is larger than the partition size

[Aihua Xu (JIRA)]

Aihua Xu created HIVE-12574:
---

 Summary: windowing function returns incorrect result when the 
window size is larger than the partition size
 Key: HIVE-12574
 URL: https://issues.apache.org/jira/browse/HIVE-12574
 Project: Hive
  Issue Type: Sub-task
  Components: PTF-Windowing
Affects Versions: 2.0.0
Reporter: Aihua Xu
Assignee: Aihua Xu


In PTF windowing, when the partition is small and the window size is larger 
than the partition size, we are seeing incorrect result.

{noformat}
CREATE TABLE sdy1(
ord int,
type string);
{noformat]

The data is:
{noformat}
2 a
3 a
1 a 
{noformat}

The result is as follows for the query {{select ord, collect_set(ord) over 
(partition by type order by ord rows between 1 preceding and 7 following) }}
{noformat}
1 [1,2,3]
2 [1,2,3]
3 [1,2,3] 
{noformat}

The expected result is:
1 [1,2,3]
2 [1,2,3]
3 [2,3]
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12580) CalcitePlanner.genOpTree tries to run with cbo disable even after a semantic analysis exception

[Hari Sankar Sivarama Subramaniyan (JIRA)]

Hari Sankar Sivarama Subramaniyan created HIVE-12580:


 Summary: CalcitePlanner.genOpTree tries to run with cbo disable 
even after a semantic analysis exception
 Key: HIVE-12580
 URL: https://issues.apache.org/jira/browse/HIVE-12580
 Project: Hive
  Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan


The following code in CalcitePlanner.genOPTree looks buggy : 

{code}
  if (!conf.getBoolVar(ConfVars.HIVE_IN_TEST) || isMissingStats
  || e instanceof CalciteSemanticException) {
reAnalyzeAST = true;
  } else if (e instanceof SemanticException) {
throw (SemanticException) e;
  } else if (e instanceof RuntimeException) {
throw (RuntimeException) e;
  } else {
throw new SemanticException(e);
  }
{code}

If we have missing stats and a Semantic exception thrown from from the try 
block, we still go ahead and continue to generate a plan with cbo turned off.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12579) add support for datanucleus.connectionPoolingType=None in TxnHandler.setupJdbcConnectionPool()

[Eugene Koifman (JIRA)]

Eugene Koifman created HIVE-12579:
-

 Summary: add support for datanucleus.connectionPoolingType=None in 
TxnHandler.setupJdbcConnectionPool()
 Key: HIVE-12579
 URL: https://issues.apache.org/jira/browse/HIVE-12579
 Project: Hive
  Issue Type: Improvement
  Components: Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman


"None" is a valid option for datanucleus.connectionPoolingType
http://www.datanucleus.org/products/accessplatform_2_2/rdbms/connection_pooling.html#Manual.

TxnHandler.setupJdbcConnectionPool() doesn't support it.
If nothing else, this is useful for debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12581) Concurrent calls to BoneCP.getConnection() lock up

[Eugene Koifman (JIRA)]

Eugene Koifman created HIVE-12581:
-

 Summary: Concurrent calls to BoneCP.getConnection() lock up
 Key: HIVE-12581
 URL: https://issues.apache.org/jira/browse/HIVE-12581
 Project: Hive
  Issue Type: Bug
  Components: Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman


{noformat}
Thread 31078: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=226 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=68, line=2082 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.poll(long, 
java.util.concurrent.TimeUnit) @bci=62, line=467 (Interpreted frame)
 - com.jolbox.bonecp.DefaultConnectionStrategy.getConnectionInternal() @bci=50, 
line=82 (Interpreted frame)
 - com.jolbox.bonecp.AbstractConnectionStrategy.getConnection() @bci=6, line=90 
(Interpreted frame)
 - com.jolbox.bonecp.BoneCP.getConnection() @bci=4, line=553 (Interpreted frame)
 - com.jolbox.bonecp.BoneCPDataSource.getConnection() @bci=108, line=131 
(Interpreted frame)
 - org.apache.hadoop.hive.metastore.txn.TxnHandler.getDbConn(int) @bci=16, 
line=973 (Interpreted frame)
 - 
org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(org.apache.hadoop.hive.metastore.api.CheckLockRequest)
 @bci=88, line=530 (Interpreted frame)
 {noformat}

Sometimes these lock up, i.e. some sort of deadlock happens inside the 
connection pool



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 40897: HIVE-12566: Incorrect result returns when using COALESCE in WHERE condition with LEFT JOIN

[Chaoyu Tang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40897/
---

Review request for hive.


Bugs: HIVE-12566
https://issues.apache.org/jira/browse/HIVE-12566


Repository: hive-git


Description
---

The pushing down of where clause to join condition should not be used in outer 
join. Otherwise, the initially matched rows from two joined tables might no 
longer match with more join conditions, and an extra row will result in some 
outer join cases. This patch is to disable the wher clasue pushdown in outer 
join.


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 69bb9d7 
  ql/src/test/queries/clientpositive/join_cond_pushdown_unqual5.q PRE-CREATION 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual5.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/40897/diff/


Testing
---

1. Manaual test cases passed and compared with other DBs like Oracle, MySQL
2. Precommit tests


Thanks,

Chaoyu Tang

Review Request 40887: HIVE-12574: windowing function returns incorrect result when the window size is larger than the partition size

[Aihua Xu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40887/
---

Review request for hive.


Repository: hive-git


Description
---

HIVE-12574: windowing function returns incorrect result when the window size is 
larger than the partition size


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
9caffb6acc4b0b698ea974d7d7edba1b06575de3 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFFirstValue.java 
160ce915742bf2de845c8146d4f3c2dc97967692 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFLastValue.java 
f917621ba391dfb8efe72174c9f89cb014cfc65c 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFMax.java 
98abd5cc9f4048a94882fed7b3ef4870b524f122 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStreamingEvaluator.java
 3c76404276e157eb140814795b000d378d19d94f 
  ql/src/test/queries/clientpositive/windowing_windowspec4.q PRE-CREATION 
  ql/src/test/results/clientpositive/windowing_windowspec4.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/40887/diff/


Testing
---


Thanks,

Aihua Xu

unassigning the JIRAs from 2.0 release and creating a branch

[Sergey Shelukhin]

Hi.
1) I have unassigned the JIRAs from 2.0 release unless they are
critical/blocking bugs, or look like they are about to be committed.
They now target 2.1 release.
We have a few bugs to go thru, so there may be time to get other JIRAs in
even if they don’t have 2.0 specified as the target.

2) I am also going to create a branch for 2.0 release shortly, probably
after HIVE-12576 (fixing license headers) is committed.
I will send an update when the branch is created, so that the Fix version
on the JIRAs committed to master after that could be set accordingly.

[jira] [Created] (HIVE-12578) Hive query failing with error ClassCastException org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc cannot be cast to org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc

[Jesus Camacho Rodriguez (JIRA)]

Jesus Camacho Rodriguez created HIVE-12578:
--

 Summary: Hive query failing with error ClassCastException 
org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc cannot be cast to 
org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc
 Key: HIVE-12578
 URL: https://issues.apache.org/jira/browse/HIVE-12578
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


Two tables:

{noformat}
CREATE TABLE table_1 (boolean_col_1 BOOLEAN, float_col_2 FLOAT, bigint_col_3 
BIGINT, varchar0111_col_4 VARCHAR(111), bigint_col_5 BIGINT, float_col_6 FLOAT, 
boolean_col_7 BOOLEAN, decimal0101_col_8 DECIMAL(1, 1), decimal0904_col_9 
DECIMAL(9, 4), char0112_col_10 CHAR(112), double_col_11 DOUBLE, boolean_col_12 
BOOLEAN, double_col_13 DOUBLE, varchar0142_col_14 VARCHAR(142), 
timestamp_col_15 TIMESTAMP, decimal0502_col_16 DECIMAL(5, 2), smallint_col_25 
SMALLINT, decimal3222_col_18 DECIMAL(32, 22), boolean_col_19 BOOLEAN, 
decimal2012_col_20 DECIMAL(20, 12), char0204_col_21 CHAR(204), double_col_61 
DOUBLE, timestamp_col_23 TIMESTAMP, int_col_24 INT, float_col_25 FLOAT, 
smallint_col_26 SMALLINT, double_col_27 DOUBLE, char0180_col_28 CHAR(180), 
decimal1503_col_29 DECIMAL(15, 3), timestamp_col_30 TIMESTAMP, smallint_col_31 
SMALLINT, decimal2020_col_32 DECIMAL(20, 20), timestamp_col_33 TIMESTAMP, 
boolean_col_34 BOOLEAN, decimal3025_col_35 DECIMAL(30, 25), decimal3117_col_36 
DECIMAL(31, 17), timestamp_col_37 TIMESTAMP, varchar0146_col_38 VARCHAR(146), 
boolean_col_39 BOOLEAN, double_col_40 DOUBLE, float_col_41 FLOAT, 
timestamp_col_42 TIMESTAMP, double_col_43 DOUBLE, boolean_col_44 BOOLEAN, 
timestamp_col_45 TIMESTAMP, tinyint_col_8 TINYINT, int_col_47 INT, 
decimal0401_col_48 DECIMAL(4, 1), varchar0064_col_49 VARCHAR(64), string_col_50 
STRING, double_col_51 DOUBLE, string_col_52 STRING, boolean_col_53 BOOLEAN, 
int_col_54 INT, boolean_col_55 BOOLEAN, string_col_56 STRING, double_col_57 
DOUBLE, varchar0131_col_58 VARCHAR(131), boolean_col_59 BOOLEAN, bigint_col_22 
BIGINT, char0184_col_61 CHAR(184), varchar0173_col_62 VARCHAR(173), 
timestamp_col_63 TIMESTAMP, decimal1709_col_26 DECIMAL(20, 5), timestamp_col_65 
TIMESTAMP, timestamp_col_66 TIMESTAMP, timestamp_col_67 TIMESTAMP, 
boolean_col_68 BOOLEAN, decimal1208_col_20 DECIMAL(33, 11), decimal1605_col_70 
DECIMAL(16, 5), varchar0010_col_71 VARCHAR(10), tinyint_col_72 TINYINT, 
timestamp_col_10 TIMESTAMP, decimal2714_col_74 DECIMAL(27, 14), double_col_75 
DOUBLE, boolean_col_76 BOOLEAN, double_col_77 DOUBLE, string_col_78 STRING, 
boolean_col_79 BOOLEAN, boolean_col_80 BOOLEAN, decimal0803_col_81 DECIMAL(8, 
3), decimal1303_col_82 DECIMAL(13, 3), tinyint_col_83 TINYINT, 
decimal3424_col_84 DECIMAL(34, 24), float_col_85 FLOAT, boolean_col_86 BOOLEAN, 
char0233_col_87 CHAR(233));

CREATE TABLE table_18 (timestamp_col_1 TIMESTAMP, double_col_2 DOUBLE, 
boolean_col_3 BOOLEAN, timestamp_col_4 TIMESTAMP, decimal2103_col_5 DECIMAL(21, 
3), char0221_col_6 CHAR(221), tinyint_col_7 TINYINT, float_col_8 FLOAT, 
int_col_2 INT, timestamp_col_10 TIMESTAMP, char0228_col_11 CHAR(228), 
timestamp_col_12 TIMESTAMP, double_col_13 DOUBLE, tinyint_col_6 TINYINT, 
tinyint_col_33 TINYINT, smallint_col_38 SMALLINT, boolean_col_17 BOOLEAN, 
double_col_18 DOUBLE, boolean_col_19 BOOLEAN, bigint_col_20 BIGINT, 
decimal0504_col_37 DECIMAL(37, 34), boolean_col_22 BOOLEAN, double_col_23 
DOUBLE, timestamp_col_24 TIMESTAMP, varchar0076_col_25 VARCHAR(76), 
timestamp_col_18 TIMESTAMP, boolean_col_27 BOOLEAN, decimal1611_col_22 
DECIMAL(37, 5), boolean_col_29 BOOLEAN);
{noformat}

Query:
{noformat}
SELECT
COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY (t2.int_col_2 
+ t1.smallint_col_25) ORDER BY (t2.int_col_2 + t1.smallint_col_25), 
FLOOR(t1.double_col_61) DESC), 524) AS int_col,
(t2.int_col_2) + (t1.smallint_col_25) AS int_col_1,
FLOOR(t1.double_col_61) AS float_col,
COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_2 + 
t1.smallint_col_25) ORDER BY (t2.int_col_2 + t1.smallint_col_25) DESC, 
FLOOR(t1.double_col_61) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 
FOLLOWING), 704) AS int_col_2
FROM table_1 t1
INNER JOIN table_18 t2 ON (((t2.tinyint_col_6) = (t1.bigint_col_22)) AND 
((t2.decimal0504_col_37) = (t1.decimal1709_col_26))) AND ((t2.tinyint_col_33) = 
(t1.tinyint_col_8))
WHERE
(t2.smallint_col_38) IN (SELECT
COALESCE(-92, -994) AS int_col
FROM table_1 tt1
INNER JOIN table_18 tt2 ON (tt2.decimal1611_col_22) = (tt1.decimal1208_col_20)
WHERE
(t1.timestamp_col_10) = (tt2.timestamp_col_18));
{noformat}

We get the following error:
{noformat}
ClassCastException org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc cannot 
be cast to org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc
{noformat}

We need to add support for constants in Select clause of semijoin subquery.

Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40865/
---

Review request for hive and John Pullokkaran.


Repository: hive-git


Description
---

HIVE-11927


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexExecutorImpl.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveReduceExpressionsRule.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
 1f5d919 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
 d026e58 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
 631a4ca 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 2825f77 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java c005b1a 
  ql/src/test/queries/clientpositive/cbo_const.q PRE-CREATION 
  ql/src/test/queries/clientpositive/constantfolding.q PRE-CREATION 
  ql/src/test/results/clientpositive/annotate_stats_deep_filters.q.out 20ccda5 
  ql/src/test/results/clientpositive/archive_multi.q.out 5f66176 
  ql/src/test/results/clientpositive/bucketpruning1.q.out d11239f 
  ql/src/test/results/clientpositive/cast1.q.out 0bdecba 
  ql/src/test/results/clientpositive/cbo_const.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cbo_rp_cross_product_check_2.q.out a8080b6 
  ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out 09b981b 
  ql/src/test/results/clientpositive/constantfolding.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/constprog_partitioner.q.out 9bf47c2 
  ql/src/test/results/clientpositive/create_genericudf.q.out 586f0ba 
  ql/src/test/results/clientpositive/cross_product_check_1.q.out f0fdfd3 
  ql/src/test/results/clientpositive/cross_product_check_2.q.out 57d1498 
  ql/src/test/results/clientpositive/cte_2.q.out a8bc760 
  ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 24ac550 
  ql/src/test/results/clientpositive/explain_logical.q.out 739de02 
  ql/src/test/results/clientpositive/filter_cond_pushdown.q.out 99eb3f7 
  ql/src/test/results/clientpositive/flatten_and_or.q.out 9c51ff3 
  ql/src/test/results/clientpositive/groupby_ppd.q.out 6164a26 
  ql/src/test/results/clientpositive/input42.q.out 2974159 
  ql/src/test/results/clientpositive/input8.q.out 03857fc 
  ql/src/test/results/clientpositive/input_part1.q.out d6f4d3e 
  ql/src/test/results/clientpositive/input_part5.q.out f2d7335 
  ql/src/test/results/clientpositive/input_part6.q.out fa51cdf 
  ql/src/test/results/clientpositive/insert1.q.out 49dd2d5 
  ql/src/test/results/clientpositive/interval_1.q.out ae2471b 
  ql/src/test/results/clientpositive/join42.q.out dda95d9 
  ql/src/test/results/clientpositive/lineage2.q.out 83785f2 
  ql/src/test/results/clientpositive/lineage3.q.out fb5e9df 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 
7f32108 
  ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out d46b0ae 
  ql/src/test/results/clientpositive/literal_decimal.q.out eddc1a4 
  ql/src/test/results/clientpositive/load_dyn_part14.q.out b35cfaf 
  ql/src/test/results/clientpositive/mergejoin.q.out e4a9e5b 
  ql/src/test/results/clientpositive/num_op_type_conv.q.out 013a153 
  ql/src/test/results/clientpositive/orc_predicate_pushdown.q.out 2642a80 
  ql/src/test/results/clientpositive/parquet_predicate_pushdown.q.out fbd3491 
  ql/src/test/results/clientpositive/partition_multilevels.q.out c1c8778 
  ql/src/test/results/clientpositive/pcr.q.out 4e23917 
  ql/src/test/results/clientpositive/pcs.q.out 5cf0dff 
  ql/src/test/results/clientpositive/pointlookup.q.out a99b388 
  ql/src/test/results/clientpositive/pointlookup2.q.out b6270b3 
  ql/src/test/results/clientpositive/pointlookup3.q.out a37e079 
  ql/src/test/results/clientpositive/pointlookup4.q.out 157aea6 
  ql/src/test/results/clientpositive/ppd_constant_expr.q.out 17e2bab 
  ql/src/test/results/clientpositive/ppd_gby_join.q.out e3f71e7 
  ql/src/test/results/clientpositive/ppd_join.q.out 58c4e43 
  ql/src/test/results/clientpositive/ppd_join2.q.out e99839e 
  ql/src/test/results/clientpositive/ppd_join3.q.out f2b0b60 
  ql/src/test/results/clientpositive/ppd_join_filter.q.out b50d936 
  ql/src/test/results/clientpositive/ppd_outer_join5.q.out 8d13475 
  ql/src/test/results/clientpositive/ppd_repeated_alias.q.out 29113e2 
  ql/src/test/results/clientpositive/ppd_union_view.q.out 5ec1200 
  ql/src/test/results/clientpositive/quotedid_basic.q.out 50c83a8 
  ql/src/test/results/clientpositive/quotedid_partition.q.out bc52c82 
  ql/src/test/results/clientpositive/rand_partitionpruner3.q.out 634e171 
  

[jira] [Created] (HIVE-12565) VectorUDAFCount.aggregateInputSelection does not handle isRepeated case

[Matt McCline (JIRA)]

Matt McCline created HIVE-12565:
---

 Summary: VectorUDAFCount.aggregateInputSelection does not handle 
isRepeated case
 Key: HIVE-12565
 URL: https://issues.apache.org/jira/browse/HIVE-12565
 Project: Hive
  Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline


Not handling the isRepeated case causes wrong results when column is null.

(Native Vector MapJoin OUTER JOIN operators will generate isRepeated null 
columns).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 40867: HIVE-11527 - bypass HiveServer2 thrift interface for query results

[Takanobu Asanuma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40867/
---

Review request for hive.


Repository: hive-git


Description
---

This is a WIP patch for HIVE-11527

* I added a new configuration whose name is 
hive.server2.webhdfs.bypass.enabled. The default is false. When this value is 
true, clients use the bypass.

* I still have not considered security such as Kerberos and SSL at present.

* I have not implement Statement#setFetchSize for bypass yet.


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java db942b0 
  jdbc/src/java/org/apache/hive/jdbc/HiveQueryResultSet.java 245c6a3 
  jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java 180f99e8 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 8fafd61 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java 1634143 
  service/if/TCLIService.thrift baf583f 
  service/src/gen/thrift/gen-cpp/TCLIService_types.h b078c99 
  service/src/gen/thrift/gen-cpp/TCLIService_types.cpp b852379 
  
service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementResp.java
 0b9aa0f 
  
service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TProtocolVersion.java
 c936ada 
  service/src/gen/thrift/gen-py/TCLIService/ttypes.py ef5f5f5 
  service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote e167d5b 
  service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb f004ec4 
  service/src/java/org/apache/hive/service/cli/CLIService.java adc9809 
  service/src/java/org/apache/hive/service/cli/operation/Operation.java 25cefc2 
  service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
b0bd351 
  service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
1331a99 
  service/src/java/org/apache/hive/service/cli/session/HiveSession.java 4f4e92d 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
a14908b 
  service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
8434965 

Diff: https://reviews.apache.org/r/40867/diff/


Testing
---

I have tested few simple queries and they worked well. But I think there are 
some problems for some queries. I'm going to test more queries and fix bugs. 
I'm also going to add unit tests.


Thanks,

Takanobu Asanuma

[jira] [Created] (HIVE-12576) Add missing license headers in preparation for the 2.0 release

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-12576:
---

 Summary: Add missing license headers in preparation for the 2.0 
release
 Key: HIVE-12576
 URL: https://issues.apache.org/jira/browse/HIVE-12576
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

branch-2.0 was created

[Sergey Shelukhin]

branch-2.0 has been created as of git commit
bbe23403cf34e27af04a271c49d484d2fbbd8656. Please use 2.1 fixVersion for
master commits and backport JIRAs that should go into 2.0 release to the
release branch.


I will update the SNAPSHOT version on master shortly.