[jira] [Created] (HIVE-14363) bucketmap inner join query fails due to NullPointerException in some cases
Jagruti Varia created HIVE-14363:
Summary: bucketmap inner join query fails due to
NullPointerException in some cases
Key: HIVE-14363
URL: https://issues.apache.org/jira/browse/HIVE-14363
Project: Hive
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Jagruti Varia
Assignee: Hari Sankar Sivarama Subramaniyan
Bucketmap inner join query between bucketed tables throws following exception
when one table contains all the empty buckets while other has all the non-empty
buckets.
{noformat}
Vertex failed, vertexName=Map 2, vertexId=vertex_1466710232033_0432_4_01,
diagnostics=[Task failed, taskId=task_1466710232033_0432_4_01_00,
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
failure ) :
attempt_1466710232033_0432_4_01_00_0:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:330)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:184)
... 14 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getKeyValueReader(MapRecordProcessor.java:372)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.initializeMapRecordSources(MapRecordProcessor.java:344)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:292)
... 15 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
attempt_1466710232033_0432_4_01_00_1:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:330)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:184)
... 14 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getKeyValueReader(MapRecordProcessor.java:372)
at
[jira] [Created] (HIVE-14244) bucketmap right outer join query throws ArrayIndexOutOfBoundsException
Jagruti Varia created HIVE-14244:
Summary: bucketmap right outer join query throws
ArrayIndexOutOfBoundsException
Key: HIVE-14244
URL: https://issues.apache.org/jira/browse/HIVE-14244
Project: Hive
Issue Type: Bug
Affects Versions: 2.1.0, 1.3.0
Reporter: Jagruti Varia
Assignee: Zhiyuan Yang
bucketmap right outer join on partitioned bucketed table throws this error:
{noformat}
Vertex failed, vertexName=Map 1, vertexId=vertex_1466710232033_0539_6_00,
diagnostics=[Task failed, taskId=task_1466710232033_0539_6_00_00,
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
failure ) :
attempt_1466710232033_0539_6_00_00_0:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:393)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:850)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.process(ReduceSinkOperator.java:416)
at
org.apache.hadoop.hive.ql.exec.vector.VectorReduceSinkOperator.process(VectorReduceSinkOperator.java:104)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:879)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:762)
... 18 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:314)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:257)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor$TezKVOutputCollector.collect(TezProcessor.java:253)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.collect(ReduceSinkOperator.java:552)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.process(ReduceSinkOperator.java:398)
... 22 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
attempt_1466710232033_0539_6_00_00_1:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
[jira] [Created] (HIVE-14076) "Vectorizaton is not supported for datatype:VOID" error while inserting data into specific columns
Jagruti Varia created HIVE-14076:
Summary: "Vectorizaton is not supported for datatype:VOID" error
while inserting data into specific columns
Key: HIVE-14076
URL: https://issues.apache.org/jira/browse/HIVE-14076
Project: Hive
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Jagruti Varia
Assignee: Jesus Camacho Rodriguez
Insert into specific columns fails due to following error:
{noformat}
Vertex failed, vertexName=Reducer 2, vertexId=vertex_1465261180142_0160_1_01,
diagnostics=[Task failed, taskId=task_1465261180142_0160_1_01_00,
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
failure ) :
attempt_1465261180142_0160_1_01_00_0:java.lang.RuntimeException:
java.lang.RuntimeException: Reduce operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:198)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Reduce operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.init(ReduceRecordSource.java:221)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.initializeSourceForTag(ReduceRecordProcessor.java:245)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:163)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
... 14 more
Caused by: java.lang.RuntimeException: Vectorizaton is not supported for
datatype:VOID
at
org.apache.hadoop.hive.ql.exec.vector.VectorizedBatchUtil.createColumnVector(VectorizedBatchUtil.java:172)
at
org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatchCtx.createVectorizedRowBatch(VectorizedRowBatchCtx.java:194)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.init(ReduceRecordSource.java:177)
... 17 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
attempt_1465261180142_0160_1_01_00_1:java.lang.RuntimeException:
java.lang.RuntimeException: Reduce operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:198)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Reduce operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.init(ReduceRecordSource.java:221)
at
[jira] [Created] (HIVE-14022) left semi join throws SemanticException if where clause contains columnname with table alias
Jagruti Varia created HIVE-14022:
Summary: left semi join throws SemanticException if where clause
contains columnname with table alias
Key: HIVE-14022
URL: https://issues.apache.org/jira/browse/HIVE-14022
Project: Hive
Issue Type: Bug
Reporter: Jagruti Varia
Assignee: Jesus Camacho Rodriguez
Fix For: 2.2.0
Left semi join throws following error if where clause contains column name with
table alias
{noformat}
select * from src_emptybucket_partitioned_1 e1 left semi join
src_emptybucket_partitioned_3 e3 on e1.age = e3.age where e1.year = 2015 and
e3.year1=2016;
16/06/10 22:37:37 [main]: INFO log.PerfLogger:
16/06/10 22:37:37 [main]: INFO log.PerfLogger:
16/06/10 22:37:37 [main]: INFO log.PerfLogger:
16/06/10 22:37:37 [main]: INFO ql.Driver: We are setting the hadoop caller
context from to hrt_qa_20160610223737_c3821398-d8df-44d8-9dd5-e66c9b7ed7c7
16/06/10 22:37:37 [main]: DEBUG parse.VariableSubstitution: Substitution is on:
select * from src_emptybucket_partitioned_1 e1 left semi join
src_emptybucket_partitioned_3 e3 on e1.age = e3.age where e1.year = 2015 and
e3.year1=2016
16/06/10 22:37:37 [main]: INFO log.PerfLogger:
16/06/10 22:37:37 [main]: INFO parse.ParseDriver: Parsing command: select *
from src_emptybucket_partitioned_1 e1 left semi join
src_emptybucket_partitioned_3 e3 on e1.age = e3.age where e1.year = 2015 and
e3.year1=2016
16/06/10 22:37:37 [main]: INFO parse.ParseDriver: Parse Completed
16/06/10 22:37:37 [main]: INFO log.PerfLogger:
16/06/10 22:37:37 [main]: DEBUG ql.Driver: Encoding valid txns info
9223372036854775807:
16/06/10 22:37:37 [main]: INFO log.PerfLogger:
16/06/10 22:37:37 [main]: INFO parse.CalcitePlanner: Starting Semantic Analysis
16/06/10 22:37:37 [main]: INFO parse.CalcitePlanner: Completed phase 1 of
Semantic Analysis
16/06/10 22:37:37 [main]: INFO parse.CalcitePlanner: Get metadata for source
tables
16/06/10 22:37:37 [main]: INFO parse.CalcitePlanner: Get metadata for subqueries
16/06/10 22:37:37 [main]: INFO parse.CalcitePlanner: Get metadata for
destination tables
16/06/10 22:37:37 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC
Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa sending #194
16/06/10 22:37:37 [IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa]: DEBUG
ipc.Client: IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa got value #194
16/06/10 22:37:37 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getEZForPath took
2ms
16/06/10 22:37:37 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC
Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa sending #195
16/06/10 22:37:37 [IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa]: DEBUG
ipc.Client: IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa got value #195
16/06/10 22:37:37 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getEZForPath took
1ms
16/06/10 22:37:37 [main]: DEBUG hdfs.DFSClient:
/tmp/hive/hrt_qa/d2568b75-6399-46df-82b9-34ec445e8f64/hive_2016-06-10_22-37-37_392_2780828105665881901-1:
masked=rwx--
16/06/10 22:37:37 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC
Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa sending #196
16/06/10 22:37:37 [IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa]: DEBUG
ipc.Client: IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa got value #196
16/06/10 22:37:37 [main]: DEBUG ipc.ProtobufRpcEngine: Call: mkdirs took 2ms
16/06/10 22:37:37 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC
Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa sending #197
16/06/10 22:37:37 [IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa]: DEBUG
ipc.Client: IPC Client (147022238) connection to
jvaria-hive2-440-5.openstacklocal/172.22.126.47:8020 from hrt_qa got value #197
16/06/10 22:37:37 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took
1ms
16/06/10 22:37:37 [main]: INFO ql.Context: New scratch dir is
hdfs://jvaria-hive2-440-5.openstacklocal:8020/tmp/hive/hrt_qa/d2568b75-6399-46df-82b9-34ec445e8f64/hive_2016-06-10_22-37-37_392_2780828105665881901-1
16/06/10 22:37:37 [main]: INFO parse.CalcitePlanner: Completed getting MetaData
in Semantic Analysis
16/06/10 22:37:37 [main]: INFO parse.BaseSemanticAnalyzer: Not invoking CBO
because the statement has too few joins
16/06/10 22:37:37 [main]: DEBUG hive.log: DDL: struct
[jira] [Created] (HIVE-12012) select query on json table with map type column fails
Jagruti Varia created HIVE-12012:
Summary: select query on json table with map type column fails
Key: HIVE-12012
URL: https://issues.apache.org/jira/browse/HIVE-12012
Project: Hive
Issue Type: Bug
Reporter: Jagruti Varia
Assignee: Jason Dere
select query on json table throws this error if table contains map type column:
{noformat}
Failed with exception
java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@295f79b; line: 1, column: 26]
{noformat}
steps to reproduce the issue:
{noformat}
hive> create table c_complex(a array,b map) row format
serde 'org.apache.hive.hcatalog.data.JsonSerDe';
OK
Time taken: 0.319 seconds
hive> insert into table c_complex select array('aaa'),map('aaa',1) from
studenttab10k limit 2;
Query ID = hrt_qa_20150826183232_47deb33a-19c0-4d2b-a92f-726659eb9413
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1440603993714_0010)
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
Map 1 .. SUCCEEDED 1 100 0 0
Reducer 2 .. SUCCEEDED 1 100 0 0
VERTICES: 02/02 [==>>] 100% ELAPSED TIME: 11.75 s
Loading data to table default.c_complex
Table default.c_complex stats: [numFiles=1, numRows=2, totalSize=56,
rawDataSize=0]
OK
Time taken: 13.706 seconds
hive> select * from c_complex;
OK
Failed with exception
java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@295f79b; line: 1, column: 26]
Time taken: 0.115 seconds
hive> select count(*) from c_complex;
OK
2
Time taken: 0.205 seconds, Fetched: 1 row(s)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-10885) with vectorization enabled join operation involving interval_day_time fails
Jagruti Varia created HIVE-10885:
Summary: with vectorization enabled join operation involving
interval_day_time fails
Key: HIVE-10885
URL: https://issues.apache.org/jira/browse/HIVE-10885
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Jagruti Varia
Assignee: Matt McCline
When vectorization is on, join operation involving interval_day_time type
throws following error:
{noformat}
Status: Failed
Vertex failed, vertexName=Map 2, vertexId=vertex_1432858236614_0247_1_01,
diagnostics=[Task failed, taskId=task_1432858236614_0247_1_01_00,
diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running
task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator
initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:229)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:147)
... 14 more
Caused by: java.lang.RuntimeException: Cannot allocate vector copy row for
interval_day_time
at
org.apache.hadoop.hive.ql.exec.vector.VectorCopyRow.init(VectorCopyRow.java:213)
at
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinCommonOperator.initializeOp(VectorMapJoinCommonOperator.java:581)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:362)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:438)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:214)
... 15 more
], TaskAttempt 1 failed, info=[Error: Failure while running
task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator
initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:229)
at
[jira] [Created] (HIVE-10861) can't use trunc() udf without specifying format
Jagruti Varia created HIVE-10861:
Summary: can't use trunc() udf without specifying format
Key: HIVE-10861
URL: https://issues.apache.org/jira/browse/HIVE-10861
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Jagruti Varia
Assignee: Jason Dere
From the description, it seems like format argument can be omitted while using
trunc() udf. But it throws an error if only date argument is provided.
{noformat}
hive desc function trunc;
OK
trunc(date, fmt) - Returns returns date with the time portion of the day
truncated to the unit specified by the format model fmt. If you omit fmt, then
date is truncated to the nearest day. It now only supports 'MONTH'/'MON'/'MM'
and 'YEAR'/''/'YY' as format.
Time taken: 0.019 seconds, Fetched: 1 row(s)
hive select trunc('2015-12-11');
FAILED: SemanticException [Error 10015]: Line 1:7 Arguments length mismatch
''2015-12-11'': trunc() requires 2 argument, got 1
{noformat}
UDF wiki
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-10688) constant folding is broken for case-when udf
Jagruti Varia created HIVE-10688:
Summary: constant folding is broken for case-when udf
Key: HIVE-10688
URL: https://issues.apache.org/jira/browse/HIVE-10688
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Jagruti Varia
Assignee: Ashutosh Chauhan
Fix For: 1.2.0
In some cases, case-when udf throws IndexOutOfBoundsException as shown below:
{noformat}
FAILED: IndexOutOfBoundsException Index: 2, Size: 2
java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.shortcutFunction(ConstantPropagateProcFactory.java:428)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:238)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:227)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.access$000(ConstantPropagateProcFactory.java:98)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory$ConstantPropagateFilterProc.process(ConstantPropagateProcFactory.java:679)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:95)
[jira] [Created] (HIVE-10679) JsonSerde ignores varchar and char size limit specified during table creation
Jagruti Varia created HIVE-10679:
Summary: JsonSerde ignores varchar and char size limit specified
during table creation
Key: HIVE-10679
URL: https://issues.apache.org/jira/browse/HIVE-10679
Project: Hive
Issue Type: Bug
Reporter: Jagruti Varia
Assignee: Jason Dere
JsonSerde ignores varchar and char size limit specified during table creation
and always creates varchar or char column with max length.
steps to reproduce the issue:
{noformat}
create table jsonserde_1 (v varchar(50), c char(50)) row format serde
'org.apache.hive.hcatalog.data.JsonSerDe';
desc jsonserde_1;
OK
v varchar(65535) from deserializer
c char(255) from deserializer
Time taken: 0.468 seconds, Fetched: 2 row(s)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7947) Add message at the end of each testcase with timestamp in Webhcat system tests
[
https://issues.apache.org/jira/browse/HIVE-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120845#comment-14120845
]
Jagruti Varia commented on HIVE-7947:
-
Thanks all for reviewing and merging this change.
Add message at the end of each testcase with timestamp in Webhcat system tests
--
Key: HIVE-7947
URL: https://issues.apache.org/jira/browse/HIVE-7947
Project: Hive
Issue Type: Improvement
Components: Tests, WebHCat
Reporter: Jagruti Varia
Assignee: Jagruti Varia
Priority: Trivial
Fix For: 0.14.0
Attachments: HIVE-7947.1.patch
Currently, Webhcat e2e testsuite only prints message while starting test run:
{noformat}
Beginning test testcase at 1406716992
{noformat}
It should also print ending message with timestamp similar to this:
{noformat}
Ending test testcase at 1406717992
{noformat}
This change will make log collection easy for failed test cases.
NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-7947) Add message at the end of each testcase with timestamp in Webhcat system tests
Jagruti Varia created HIVE-7947:
---
Summary: Add message at the end of each testcase with timestamp in
Webhcat system tests
Key: HIVE-7947
URL: https://issues.apache.org/jira/browse/HIVE-7947
Project: Hive
Issue Type: Improvement
Components: Tests, WebHCat
Reporter: Jagruti Varia
Priority: Trivial
Fix For: 0.14.0
Currently, Webhcat e2e testsuite only prints message while starting test run:
{noformat}
Beginning test testcase at 1406716992
{noformat}
It should also print ending message with timestamp similar to this:
{noformat}
Ending test testcase at 1406717992
{noformat}
This change will make log collection easy for failed test cases.
NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7947) Add message at the end of each testcase with timestamp in Webhcat system tests
[
https://issues.apache.org/jira/browse/HIVE-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jagruti Varia updated HIVE-7947:
Attachment: HIVE-7947.1.patch
Attaching a patch for review.
Add message at the end of each testcase with timestamp in Webhcat system tests
--
Key: HIVE-7947
URL: https://issues.apache.org/jira/browse/HIVE-7947
Project: Hive
Issue Type: Improvement
Components: Tests, WebHCat
Reporter: Jagruti Varia
Priority: Trivial
Fix For: 0.14.0
Attachments: HIVE-7947.1.patch
Currently, Webhcat e2e testsuite only prints message while starting test run:
{noformat}
Beginning test testcase at 1406716992
{noformat}
It should also print ending message with timestamp similar to this:
{noformat}
Ending test testcase at 1406717992
{noformat}
This change will make log collection easy for failed test cases.
NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7947) Add message at the end of each testcase with timestamp in Webhcat system tests
[
https://issues.apache.org/jira/browse/HIVE-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jagruti Varia updated HIVE-7947:
Status: Patch Available (was: Open)
Add message at the end of each testcase with timestamp in Webhcat system tests
--
Key: HIVE-7947
URL: https://issues.apache.org/jira/browse/HIVE-7947
Project: Hive
Issue Type: Improvement
Components: Tests, WebHCat
Reporter: Jagruti Varia
Assignee: Jagruti Varia
Priority: Trivial
Fix For: 0.14.0
Attachments: HIVE-7947.1.patch
Currently, Webhcat e2e testsuite only prints message while starting test run:
{noformat}
Beginning test testcase at 1406716992
{noformat}
It should also print ending message with timestamp similar to this:
{noformat}
Ending test testcase at 1406717992
{noformat}
This change will make log collection easy for failed test cases.
NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
