[jira] [Created] (HIVE-18997) Hive column casting from decimal to double is resulting in NULL

[Rangaswamy Narayan (JIRA)]

Rangaswamy Narayan created HIVE-18997:
-

 Summary: Hive column casting from decimal to double is resulting 
in NULL
 Key: HIVE-18997
 URL: https://issues.apache.org/jira/browse/HIVE-18997
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 1.1.0
 Environment: i have checked this is hive CLI. 
Reporter: Rangaswamy Narayan


I have hive table table1 schema of the table looks like this

{{[CREATE TABLE table1(p_decimal1 DECIMAL(38,5)) ROW FORMAT DELIMITED FIELDS 
TERMINATED BY ',' STORED AS TEXTFILE] }}

and i have below value in the table

{{row : col(p_decimal1) row1 : 12345123451234512345123.45123 }}

in later stage if i execute

{{select CAST(p_decimal1 AS DOUBLE) from table1; }}

query then I am getting {{NULL}} as a output. 

expected output should be non-null value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18996) SubString Druid convertor assuming that index is always constant literal value

[slim bouguerra (JIRA)]

slim bouguerra created HIVE-18996:
-

 Summary: SubString Druid convertor assuming that index is always 
constant literal value
 Key: HIVE-18996
 URL: https://issues.apache.org/jira/browse/HIVE-18996
 Project: Hive
  Issue Type: Bug
Reporter: slim bouguerra


Query like the following 
{code}
SELECT substring(namespace, CAST(deleted AS INT), 4)
FROM druid_table_1;
{code}
will fail with 
{code}
java.lang.AssertionError: not a literal: $13
at org.apache.calcite.rex.RexLiteral.findValue(RexLiteral.java:963)
at org.apache.calcite.rex.RexLiteral.findValue(RexLiteral.java:955)
at org.apache.calcite.rex.RexLiteral.intValue(RexLiteral.java:938)
at 
org.apache.calcite.adapter.druid.SubstringOperatorConversion.toDruidExpression(SubstringOperatorConversion.java:46)
at 
org.apache.calcite.adapter.druid.DruidExpressions.toDruidExpression(DruidExpressions.java:120)
at 
org.apache.calcite.adapter.druid.DruidQuery.computeProjectAsScan(DruidQuery.java:746)
at 
org.apache.calcite.adapter.druid.DruidRules$DruidProjectRule.onMatch(DruidRules.java:308)
at 
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317)
{code}

because is assuming that index is always a constant literal. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 66167: HIVE-18952 Tez session disconnect and reconnect on HS2 HA failover

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66167/
---

Review request for hive.


Repository: hive-git


Description
---

see jira


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
f8e715d96efd95a68239728aeb8b8cc79450c20c 
  
llap-client/src/java/org/apache/hadoop/hive/registry/ServiceInstanceStateChangeListener.java
 cc1ba337523e200701f4ad108cf770988548c34f 
  llap-client/src/java/org/apache/hadoop/hive/registry/impl/TezAmInstance.java 
a8629479ec8b0044595c481dfe4d9fea1f26550e 
  
llap-client/src/java/org/apache/hadoop/hive/registry/impl/TezAmRegistryImpl.java
 3ff732d9b796190e515111391223922dcff07bbd 
  llap-client/src/java/org/apache/hadoop/hive/registry/impl/ZkRegistryBase.java 
7ca3548561d5d3f0289d04c2722c4e9bcd5f0a7c 
  
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskSchedulerService.java
 82179645da00c5673bee97e7a8ae8dd917736520 
  pom.xml 5be30f669666e85225af32011abb33ddb7c17d53 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SessionExpirationTracker.java 
df14f1122651738a884c64118d140ec44aca7497 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java 
89954cba6709bc4c39af3dd7ade85bf8456aaa0e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
a051f9019572285a8ba62e6b4f9ecead860ac22c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolSession.java 
d3748edb8674f9fdf49563238a3c3524f8acefe1 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java 
046ea19c7227c16ec9c71cf470652385347cc78b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 
f0e620c684519f73c08bfa49d45d244085cc8b65 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
20a5947291f71c22d5da6ea2740426e50366344b 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 
bb92c443e2364d9925bad60b0b9df9db8fd9134d 


Diff: https://reviews.apache.org/r/66167/diff/1/


Testing
---


Thanks,

Sergey Shelukhin

[GitHub] hive pull request #322: Branch 2

[sasikanthpotru]

GitHub user sasikanthpotru opened a pull request:

https://github.com/apache/hive/pull/322

Branch 2



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/hive branch-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/322.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #322


commit 47bc1c196d8e5d06ce7bd31daa2e4c5f87aecc01
Author: Wei Zheng 
Date:   2017-03-24T16:52:52Z

HIVE-15644 : Collect LLAP's JVM metrics via Hive's JvmPauseMonitor (Wei 
Zheng, reviewed by Prasanth Jayachandran)

commit d65a2491992ffa691d3904c98559f42dbed81e60
Author: Sahil Takiar 
Date:   2017-03-24T08:38:04Z

HIVE-16276: Fix NoSuchMethodError: 
com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
 (Sahil Takiar, reviewed by Slim Bouguerra, Jesus Camacho Rodriguez)

commit e356de8e68011ab58ad7799a7ce70970e781fa27
Author: Pengcheng Xiong 
Date:   2017-03-26T00:28:01Z

HIVE-16274: Support tuning of NDV of columns using lower/upper bounds 
(Pengcheng Xiong, reviewed by Jason Dere)

commit 832b9a44cf52297d08d5a7241ebf440a67a8152e
Author: Naveen Gangam 
Date:   2017-03-26T20:58:00Z

HIVE-12274 Increase width of columns used for general configuration in the 
metastore (Naveen Gangam, Reviewed by Aihua Xu)

commit 982606e96268bf64fe036d329227b15a4976c8b6
Author: Naveen Gangam 
Date:   2017-03-27T02:23:29Z

HIVE-12274 Follow-on commit to correct a comment

commit 0273823fb1f0e3e5a453bf25968f2595a9dde37f
Author: Sahil Takiar 
Date:   2017-03-27T22:58:11Z

HIVE-16188: beeline should block the connection if given invalid database 
name. (Sahil Takiar, reviewed by Sergio Pena, Vihang Karajgaonkar, Pavas Garg)

commit d18be94984564350fb96233b9e5ae1e00186fbcf
Author: Naveen Gangam 
Date:   2017-03-28T18:19:26Z

HIVE-16007 LogRunnable never stops when fails to compile. (Peter Vary, 
Reviewed by Sergey Shelukhin, anishek, Naveen Gangam)

commit 122350053d98b7662a18bad43b60d6b3929faf65
Author: Pengcheng Xiong 
Date:   2017-03-28T18:39:22Z

HIVE-16310: Get the output operators of Reducesink when vectorization is on 
(Pengcheng Xiong, reviewed by Ashutosh Chauhan)

commit 8b866562b16a2b10880a4296fe133ef007a85c77
Author: Barna Zsombor Klara 
Date:   2017-03-28T19:03:02Z

HIVE-16231: Parquet timestamp may be stored differently since HIVE-12767 
(Barna Zsombor Klara, reviewed by Sergio Pena)

commit a1f0295732d34011ee90163c2fd91b2250d967f9
Author: Sergey Shelukhin 
Date:   2017-03-28T20:57:52Z

HIVE-15829 : LLAP text cache: disable memory tracking on the writer (Sergey 
Shelukhin, reviewed by Prasanth Jayachandran)

commit e33a50fb96f24a68c1448fa4c62c830bc8e2c1cb
Author: Sergey Shelukhin 
Date:   2017-03-28T20:59:17Z

HIVE-14077 : add implicit decimal arithmetic q test, fix issues if found 
(Sergey Shelukhin, reviewed by Ashutosh Chauhan)

commit 48f6e30080df475ffe54c39f70dd134268e30358
Author: Daniel Dai 
Date:   2017-03-28T22:31:07Z

HIVE-16305: Additional Datanucleus ClassLoaderResolverImpl leaks causing 
HS2 OOM (Daniel Dai, reviewed by Thejas Nair)

commit 9c6e01e707854714bee047b8513ca8e1ad7f4e58
Author: David Maughan 
Date:   2016-12-16T09:07:00Z

HIVE-15434 : Add UDF to allow interrogation of uniontype values (David 
Maughan via Ashutosh Chauhan)

Signed-off-by: Ashutosh Chauhan 

commit e222a076d4c1f9ca7a7605c6de46668247422769
Author: Sergey Shelukhin 
Date:   2017-03-29T19:09:48Z

HIVE-16318 : LLAP cache: address some issues in 2.2/2.3 (Sergey Shelukhin, 
reviewed by Siddharth Seth and Prasanth Jayachandran)

commit 1ff14a2779410d652f62d1429c03789c4164ffe1
Author: Thomas Poepping 
Date:   2017-03-29T17:29:46Z

HIVE-16288 : Add blobstore tests for ORC and RCFILE file formats (Thomas 
Poepping via Ashutosh Chauhan)

Signed-off-by: Ashutosh Chauhan 

commit 53d9974aaa737dea49c63d41c297e8785bd41ad9
Author: Pengcheng Xiong 
Date:   2017-03-30T18:25:06Z

HIVE-16301: Preparing for 2.3 development (Naveen Gangam, reviewed by 
Pengcheng Xiong)

commit 7df55c9325efc2a6e9cc862863bbd0f40c3d7af7
Author: Rui Li 
Date:   2017-03-31T03:39:00Z

HIVE-16315: Describe table doesn't show num of partitions (Rui Li reviewed 
by Ashutosh Chauhan)

commit e122de829d5780290454adb7fb571785814a44db
Author: Chaoyu Tang 
Date:   2017-04-01T14:49:43Z

HIVE-15880: Allow insert overwrite and truncate table query to use 
auto.purge table property (Vihang Karajgaonkar via Chaoyu Tang)

commit 4cdef08f7ee43ec2a3842e3e1926003e36761caa

Re: Review Request 66069: HIVE-18926 Imporve operator-tree matching

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66069/#review199495
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java
Lines 381 (patched)


Can you please add a comment here that what we are doing here and why?



ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
Line 2501 (original), 2507 (patched)


I think lookup(op) is better than lookup(treeSig) since that way we can 
delay computation of signature. e.g, if I configure to run with 
EmptyStatsSource (to turn off this feature), this computation will be avoided.



ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java
Line 43 (original), 48 (patched)


Add comments about this class.



ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java
Line 63 (original), 71 (patched)


Comments.


- Ashutosh Chauhan


On March 19, 2018, 9:59 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66069/
> ---
> 
> (Updated March 19, 2018, 9:59 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-18926
> https://issues.apache.org/jira/browse/HIVE-18926
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> * match operators based on opsigs
> * always calc operator sigs
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 
> a235f3fbf4 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/OpSignature.java 
> 90b2fd3dad 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/OpTreeSignature.java
>  c6d1a6aaca 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/Signature.java 
> c228a8e4f5 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/SignatureUtils.java 
> 2269322b69 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  22b052cd07 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CommonMergeJoinDesc.java 
> 5a81add706 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/JoinCondDesc.java ea22131c56 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewJoinDesc.java 
> 85a4683491 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java 91ea159d52 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java f2955af25e 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java 
> 36d7e589a8 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapperProcess.java 
> 424dd7956b 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/RuntimeStatsSource.java 
> 21a0678153 
>   
> ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SimpleRuntimeStatsSource.java
>  6f340b8450 
>   ql/src/java/org/apache/hadoop/hive/ql/reexec/ReExecDriver.java 93031712dc 
>   
> ql/src/test/org/apache/hadoop/hive/ql/optimizer/signature/TestOperatorSignature.java
>  8c899e7fef 
>   ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestOperatorCmp.java 
> cfb0ca38b2 
>   ql/src/test/results/clientpositive/llap/retry_failure_stat_changes.q.out 
> 9b58ce0e5d 
> 
> 
> Diff: https://reviews.apache.org/r/66069/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zoltan Haindrich
> 
>

[jira] [Created] (HIVE-18995) Vectorization: Add option to suppress "Execution mode: vectorized" for testing purposes

[Matt McCline (JIRA)]

Matt McCline created HIVE-18995:
---

 Summary: Vectorization: Add option to suppress "Execution mode: 
vectorized" for testing purposes
 Key: HIVE-18995
 URL: https://issues.apache.org/jira/browse/HIVE-18995
 Project: Hive
  Issue Type: Improvement
  Components: Hive
Reporter: Matt McCline
Assignee: Matt McCline


In order to see Q file differences in large runs it is helpful to eliminate 
change noise from "Execution mode: vectorized" in EXPLAIN output.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18994) Handle client connections on failover

[Prasanth Jayachandran (JIRA)]

Prasanth Jayachandran created HIVE-18994:


 Summary: Handle client connections on failover
 Key: HIVE-18994
 URL: https://issues.apache.org/jira/browse/HIVE-18994
 Project: Hive
  Issue Type: Sub-task
  Components: HiveServer2
Affects Versions: 3.0.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


When leader failover happens (either automatically or manually), tez sessions 
are closed. But client connections are not. We need to close the client 
connections explicitly so that workload manager revokes all the guaranteed 
slots and upon reconnection client will connect to active HS2 instance (this is 
to avoid clients reusing the same connection and submitting queries to passive 
HS2). In future, some timeout or other policies (may be WM will run everything 
speculatively) can be added.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18993) Use Druid Expressions

[slim bouguerra (JIRA)]

slim bouguerra created HIVE-18993:
-

 Summary: Use Druid Expressions
 Key: HIVE-18993
 URL: https://issues.apache.org/jira/browse/HIVE-18993
 Project: Hive
  Issue Type: Task
Reporter: slim bouguerra






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66131: HIVE-18983: Add support for table properties inheritance in Create table like

[Igor Kryvenko]

> On Март 19, 2018, 10:03 п.п., Vineet Garg wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
> > Lines 12767 (patched)
> > 
> >
> > Shouldn't this be under if(hivestatsautogather==false) ? Otherwise this 
> > will always end up clearing the stats properties.

I think it shouldn't be. It makes no sense.Because in DDLTask#createTableLike 
we will set up new stats properties for this table and stats from original 
table will be overrided.


- Igor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66131/#review199480
---


On Март 19, 2018, 6:57 п.п., Igor Kryvenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66131/
> ---
> 
> (Updated Март 19, 2018, 6:57 п.п.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and 
> Prasanth_J.
> 
> 
> Bugs: HIVE-18983
> https://issues.apache.org/jira/browse/HIVE-18983
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Currently, Create table like support table properties.
> But it doesn't inherit table properties from original table.
> It uses just default table properties and doesn't inherit properties from 
> original table.
> It would be great if create table like will inherit origin table properties 
> and will override them if they are specified in query .
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 2342fff2ebe7914dae4fe644c6a2182edf0ff08a 
>   ql/src/test/results/clientpositive/create_alter_list_bucketing_table1.q.out 
> e74be6221d08b7b06d56e63db9e94d6df8e0c388 
>   ql/src/test/results/clientpositive/create_like.q.out 
> 7c26a597a59de8a4861518e4279cd995446e9991 
>   ql/src/test/results/clientpositive/create_like2.q.out 
> 601b929a2d557255cbf29aa9a5ca3752eda41920 
>   ql/src/test/results/clientpositive/explain_ddl.q.out 
> 6506e89035e1f1b2b4950175f888f463183cd34d 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/common/StatsSetupConst.java
>  78ea01d9687fe043d63441430c46b30c25cd9756 
> 
> 
> Diff: https://reviews.apache.org/r/66131/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Igor Kryvenko
> 
>

Re: Review Request 66131: HIVE-18983: Add support for table properties inheritance in Create table like

[Vineet Garg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66131/#review199480
---




ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 12767 (patched)


Shouldn't this be under if(hivestatsautogather==false) ? Otherwise this 
will always end up clearing the stats properties.


- Vineet Garg


On March 19, 2018, 6:57 p.m., Igor Kryvenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66131/
> ---
> 
> (Updated March 19, 2018, 6:57 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and 
> Prasanth_J.
> 
> 
> Bugs: HIVE-18983
> https://issues.apache.org/jira/browse/HIVE-18983
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Currently, Create table like support table properties.
> But it doesn't inherit table properties from original table.
> It uses just default table properties and doesn't inherit properties from 
> original table.
> It would be great if create table like will inherit origin table properties 
> and will override them if they are specified in query .
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 2342fff2ebe7914dae4fe644c6a2182edf0ff08a 
>   ql/src/test/results/clientpositive/create_alter_list_bucketing_table1.q.out 
> e74be6221d08b7b06d56e63db9e94d6df8e0c388 
>   ql/src/test/results/clientpositive/create_like.q.out 
> 7c26a597a59de8a4861518e4279cd995446e9991 
>   ql/src/test/results/clientpositive/create_like2.q.out 
> 601b929a2d557255cbf29aa9a5ca3752eda41920 
>   ql/src/test/results/clientpositive/explain_ddl.q.out 
> 6506e89035e1f1b2b4950175f888f463183cd34d 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/common/StatsSetupConst.java
>  78ea01d9687fe043d63441430c46b30c25cd9756 
> 
> 
> Diff: https://reviews.apache.org/r/66131/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Igor Kryvenko
> 
>

Re: Review Request 66069: HIVE-18926 Imporve operator-tree matching

[Zoltan Haindrich]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66069/
---

(Updated March 19, 2018, 9:59 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

patch#4


Bugs: HIVE-18926
https://issues.apache.org/jira/browse/HIVE-18926


Repository: hive-git


Description
---

* match operators based on opsigs
* always calc operator sigs


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 
a235f3fbf4 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/OpSignature.java 
90b2fd3dad 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/OpTreeSignature.java 
c6d1a6aaca 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/Signature.java 
c228a8e4f5 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/signature/SignatureUtils.java 
2269322b69 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 22b052cd07 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CommonMergeJoinDesc.java 
5a81add706 
  ql/src/java/org/apache/hadoop/hive/ql/plan/JoinCondDesc.java ea22131c56 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewJoinDesc.java 
85a4683491 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java 91ea159d52 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java f2955af25e 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java 36d7e589a8 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapperProcess.java 
424dd7956b 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/RuntimeStatsSource.java 
21a0678153 
  
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SimpleRuntimeStatsSource.java 
6f340b8450 
  ql/src/java/org/apache/hadoop/hive/ql/reexec/ReExecDriver.java 93031712dc 
  
ql/src/test/org/apache/hadoop/hive/ql/optimizer/signature/TestOperatorSignature.java
 8c899e7fef 
  ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestOperatorCmp.java 
cfb0ca38b2 
  ql/src/test/results/clientpositive/llap/retry_failure_stat_changes.q.out 
9b58ce0e5d 


Diff: https://reviews.apache.org/r/66069/diff/2/

Changes: https://reviews.apache.org/r/66069/diff/1-2/


Testing
---


Thanks,

Zoltan Haindrich

[jira] [Created] (HIVE-18992) enable synthetic file IDs by default in LLAP

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-18992:
---

 Summary: enable synthetic file IDs by default in LLAP
 Key: HIVE-18992
 URL: https://issues.apache.org/jira/browse/HIVE-18992
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin


The file IDs are much more reliable than they were initially (hash+len+date 
instead of just one hash of everything) so they should be enabled by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18991) Drop database cascade doesn't work with materialized views

[Alan Gates (JIRA)]

Alan Gates created HIVE-18991:
-

 Summary: Drop database cascade doesn't work with materialized views
 Key: HIVE-18991
 URL: https://issues.apache.org/jira/browse/HIVE-18991
 Project: Hive
  Issue Type: Bug
  Components: Materialized views, Metastore
Affects Versions: 3.0.0
Reporter: Alan Gates


Create a database, add a table and then a materialized view that depends on the 
table.  Then drop the database with cascade set.  Sometimes this will fail 
because when HiveMetaStore.drop_database_core goes to drop all of the tables it 
may drop the base table before the materialized view, which will cause an 
integrity constraint violation in the RDBMS.  To resolve this that method 
should change to fetch and drop materialized views before tables.

cc [~jcamachorodriguez]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18990) Hive doesn't close Tez session properly

[Kryvenko Igor (JIRA)]

Kryvenko Igor created HIVE-18990:


 Summary: Hive doesn't close Tez session properly
 Key: HIVE-18990
 URL: https://issues.apache.org/jira/browse/HIVE-18990
 Project: Hive
  Issue Type: Bug
Reporter: Kryvenko Igor
Assignee: Kryvenko Igor


Hive doesn't close Tez session properly if AM isn't ready for accepting DAG.

*STR*
This can be easily reproduced using the following steps:
*1) configure cluster on Tez;*

*2) create file test.hql*
cat ~/test.hql
show databases;

*3) run the job*
$ hive --hiveconf hive.root.logger=DEBUG,console --hiveconf 
hive.execution.engine=tez -f ~/test.hql

If we login into Yarn UI,  we will see that jobs status is failed even it 
finished successfully.


It happens because hive creates tez session by default. And if query finished 
very quickly, we can't close tez session properly because AM isn't ready for 
accepting any requests.






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66131: HIVE-18983: Add support for table properties inheritance in Create table like

[Igor Kryvenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66131/
---

(Updated Март 19, 2018, 6:57 п.п.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and 
Prasanth_J.


Bugs: HIVE-18983
https://issues.apache.org/jira/browse/HIVE-18983


Repository: hive-git


Description
---

Currently, Create table like support table properties.
But it doesn't inherit table properties from original table.
It uses just default table properties and doesn't inherit properties from 
original table.
It would be great if create table like will inherit origin table properties and 
will override them if they are specified in query .


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
2342fff2ebe7914dae4fe644c6a2182edf0ff08a 
  ql/src/test/results/clientpositive/create_alter_list_bucketing_table1.q.out 
e74be6221d08b7b06d56e63db9e94d6df8e0c388 
  ql/src/test/results/clientpositive/create_like.q.out 
7c26a597a59de8a4861518e4279cd995446e9991 
  ql/src/test/results/clientpositive/create_like2.q.out 
601b929a2d557255cbf29aa9a5ca3752eda41920 
  ql/src/test/results/clientpositive/explain_ddl.q.out 
6506e89035e1f1b2b4950175f888f463183cd34d 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/common/StatsSetupConst.java
 78ea01d9687fe043d63441430c46b30c25cd9756 


Diff: https://reviews.apache.org/r/66131/diff/1/


Testing
---


Thanks,

Igor Kryvenko

Re: Review Request 66127: HIVE-18982: Provide a CLI option to manually trigger failover

[j . prasanth . j]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66127/
---

(Updated March 19, 2018, 6:52 p.m.)


Review request for hive and Sergey Shelukhin.


Changes
---

Added comments + tests.


Bugs: HIVE-18982
https://issues.apache.org/jira/browse/HIVE-18982


Repository: hive-git


Description
---

HIVE-18982: Provide a CLI option to manually trigger failover


Diffs (updated)
-

  bin/ext/hiveserver2.sh 1e94542 
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestActivePassiveHA.java 
72b2a8c 
  
service/src/java/org/apache/hive/service/server/HS2ActivePassiveHARegistry.java 
819ce19 
  service/src/java/org/apache/hive/service/server/HiveServer2.java bb92c44 
  service/src/java/org/apache/hive/service/servlet/HS2LeadershipStatus.java 
33529ed 
  service/src/java/org/apache/hive/service/servlet/HS2Peers.java a51bbeb 


Diff: https://reviews.apache.org/r/66127/diff/2/

Changes: https://reviews.apache.org/r/66127/diff/1-2/


Testing
---


Thanks,

Prasanth_J

Re: Review Request 65634: HIVE-18264: CachedStore: Store cached partitions/col stats within the table cache

[Daniel Dai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65634/#review199444
---


Ship it!




+1, let's get this in. We can open new tickets for any remaining review 
Alexander has.

- Daniel Dai


On March 9, 2018, 9 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65634/
> ---
> 
> (Updated March 9, 2018, 9 p.m.)
> 
> 
> Review request for hive, Alexander Kolbasov, Daniel Dai, and Thejas Nair.
> 
> 
> Bugs: HIVE-18264
> https://issues.apache.org/jira/browse/HIVE-18264
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-18264
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  a3725c5395 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 86c9c2b33c 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  ac71d0882f 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
>  7b44df4128 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
>  f500d63725 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java
>  f0f650ddcf 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
>  0d132f2074 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
>  32ea17495f 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java
>  50f873a013 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  75ea8c4a77 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  207d842f94 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  ab6feb6f0b 
>   standalone-metastore/src/test/resources/log4j2.properties 365687e1c9 
> 
> 
> Diff: https://reviews.apache.org/r/65634/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

[jira] [Created] (HIVE-18989) Introduce Cap on Number of Session Variables

[BELUGA BEHR (JIRA)]

BELUGA BEHR created HIVE-18989:
--

 Summary: Introduce Cap on Number of Session Variables
 Key: HIVE-18989
 URL: https://issues.apache.org/jira/browse/HIVE-18989
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 2.3.2, 3.0.0, 2.4.0
Reporter: BELUGA BEHR


Introduce a cap on the number of session variables a single client can create.

I am able to create a 1GB file with just "SET" statements and it crashes the 
HS2 with an OutOfMemory exception which keeps track of each one.

{code}
SET 6BCF5A44=DF912BD3;
SET 614DCF23=ABC24816;
SET B67951D8=DA2645BF;
SET B9596B09=AB0A2650;
SET 2D394A85=88968FC9;
SET 859AD7BE=39710968;
SET 9ED93926=75996280;
SET D547BE24=7CF95795;
SET BB0FB0BF=62DE4A1C;
SET E447EF3F=858F398F;
...
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18988) Support bootstrap replication of ACID tables

[Sankar Hariappan (JIRA)]

Sankar Hariappan created HIVE-18988:
---

 Summary: Support bootstrap replication of ACID tables
 Key: HIVE-18988
 URL: https://issues.apache.org/jira/browse/HIVE-18988
 Project: Hive
  Issue Type: Sub-task
  Components: HiveServer2, repl
Affects Versions: 3.0.0
Reporter: Sankar Hariappan
Assignee: Sankar Hariappan
 Fix For: 3.0.0


Bootstrapping of ACID tables, need special handling to replicate a stable state 
of data.
 - If ACID feature enables, then perform bootstrap dump for ACID tables with in 
read txn.
-> Dump table/partition metadata.
-> Get the list of valid data files for a table using same logic as read txn do.
-> Dump latest valid table Write ID as per current read txn.
 - Find the valid last replication state such that it points to event ID of 
open_txn event of oldest on-going txn.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66116: HIVE-18975 NPE when inserting NULL value in structure and array with HBase table

[Oleksiy Sayankin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66116/
---

(Updated March 19, 2018, 12:36 p.m.)


Review request for hive, Aihua Xu, Yongzhi Chen, and Yibing Shi.


Changes
---

Fixed code style issues


Repository: hive-git


Description
---

Initial commit


Diffs (updated)
-

  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseRowSerializer.java 
be6aa1e2f8 
  hbase-handler/src/test/queries/positive/hbase_queries.q 43efd6c7d2 
  hbase-handler/src/test/results/positive/hbase_queries.q.out 8fcd350918 


Diff: https://reviews.apache.org/r/66116/diff/2/

Changes: https://reviews.apache.org/r/66116/diff/1-2/


Testing
---


Thanks,

Oleksiy Sayankin