[jira] [Commented] (HIVE-13876) Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead

2016-06-01 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15309906#comment-15309906
 ] 

Hive QA commented on HIVE-13876:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806603/HIVE-13876.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10195 tests 
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}

Test results: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/476/testReport
Console output: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/476/console
Test logs: 
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-476/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12806603 - PreCommit-HIVE-MASTER-Build

> Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead
> ---
>
> Key: HIVE-13876
> URL: https://issues.apache.org/jira/browse/HIVE-13876
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats, Vectorization
>Affects Versions: 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Critical
> Attachments: HIVE-13876.1.patch
>
>
> High CPU usage due to exception handling code.
> {code}
>  TezTaskRunner [RUNNABLE] [DAEMON]
> java.lang.Throwable.fillInStackTrace(int) Throwable.java (native)
> java.lang.Throwable.fillInStackTrace() Throwable.java:783
> java.lang.Throwable.(String) Throwable.java:265
> java.lang.Exception.(String) Exception.java:66
> java.lang.RuntimeException.(String) RuntimeException.java:62
> java.lang.IllegalArgumentException.(String) 
> IllegalArgumentException.java:52
> java.lang.NumberFormatException.(String) NumberFormatException.java:55
> sun.misc.FloatingDecimal.readJavaFormatString(String) 
> FloatingDecimal.java:1842
> sun.misc.FloatingDecimal.parseFloat(String) FloatingDecimal.java:122
> java.lang.Float.parseFloat(String) Float.java:451
> org.apache.hadoop.hive.serde2.lazy.fast.LazySimpleDeserializeRead.readCheckNull()
>  LazySimpleDeserializeRead.java:309
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorizedRowBatch,
>  int, int) VectorDeserializeRow.java:346
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorizedRowBatch,
>  int) VectorDeserializeRow.java:659
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(Writable) 
> VectorMapOperator.java:814
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(Object) 
> MapRecordSource.java:86
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord() 
> MapRecordSource.java:70
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run() 
> MapRecordProcessor.java:361
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(Map,
>  Map) TezProcessor.java:172
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(Map, Map) 
> TezProcessor.java:160
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run() 
> LogicalIOProcessorRuntimeTask.java:370
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run() 
> TaskRunner2Callable.java:73
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run() 
> TaskRunner2Callable.java:61
> java.security.AccessController.doPrivileged(PrivilegedExceptionAction, 
> AccessControlContext) AccessController.java (native)
> javax.security.auth.Subject.doAs(Subject, PrivilegedExceptionAction) 
> Subject.java:422
> org.apache.hadoop.security.UserGroupInformation.doAs(PrivilegedExceptionAction)
>

[jira] [Commented] (HIVE-13876) Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead

2016-05-29 Thread Gopal V (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305771#comment-15305771
 ] 

Gopal V commented on HIVE-13876:


Have extended this patch to cover Date/Timestamp similarly in HIVE-13887

> Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead
> ---
>
> Key: HIVE-13876
> URL: https://issues.apache.org/jira/browse/HIVE-13876
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats, Vectorization
>Affects Versions: 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Critical
> Attachments: HIVE-13876.1.patch
>
>
> High CPU usage due to exception handling code.
> {code}
>  TezTaskRunner [RUNNABLE] [DAEMON]
> java.lang.Throwable.fillInStackTrace(int) Throwable.java (native)
> java.lang.Throwable.fillInStackTrace() Throwable.java:783
> java.lang.Throwable.(String) Throwable.java:265
> java.lang.Exception.(String) Exception.java:66
> java.lang.RuntimeException.(String) RuntimeException.java:62
> java.lang.IllegalArgumentException.(String) 
> IllegalArgumentException.java:52
> java.lang.NumberFormatException.(String) NumberFormatException.java:55
> sun.misc.FloatingDecimal.readJavaFormatString(String) 
> FloatingDecimal.java:1842
> sun.misc.FloatingDecimal.parseFloat(String) FloatingDecimal.java:122
> java.lang.Float.parseFloat(String) Float.java:451
> org.apache.hadoop.hive.serde2.lazy.fast.LazySimpleDeserializeRead.readCheckNull()
>  LazySimpleDeserializeRead.java:309
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorizedRowBatch,
>  int, int) VectorDeserializeRow.java:346
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorizedRowBatch,
>  int) VectorDeserializeRow.java:659
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(Writable) 
> VectorMapOperator.java:814
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(Object) 
> MapRecordSource.java:86
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord() 
> MapRecordSource.java:70
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run() 
> MapRecordProcessor.java:361
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(Map,
>  Map) TezProcessor.java:172
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(Map, Map) 
> TezProcessor.java:160
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run() 
> LogicalIOProcessorRuntimeTask.java:370
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run() 
> TaskRunner2Callable.java:73
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run() 
> TaskRunner2Callable.java:61
> java.security.AccessController.doPrivileged(PrivilegedExceptionAction, 
> AccessControlContext) AccessController.java (native)
> javax.security.auth.Subject.doAs(Subject, PrivilegedExceptionAction) 
> Subject.java:422
> org.apache.hadoop.security.UserGroupInformation.doAs(PrivilegedExceptionAction)
>  UserGroupInformation.java:1657
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() 
> TaskRunner2Callable.java:61
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() 
> TaskRunner2Callable.java:37
> org.apache.tez.common.CallableWithNdc.call() CallableWithNdc.java:36
> java.util.concurrent.FutureTask.run() FutureTask.java:266
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) 
> ThreadPoolExecutor.java:1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run() 
> ThreadPoolExecutor.java:617
> java.lang.Thread.run() Thread.java:745
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13876) Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead

2016-05-27 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304603#comment-15304603
 ] 

Sergey Shelukhin commented on HIVE-13876:
-

+1

> Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead
> ---
>
> Key: HIVE-13876
> URL: https://issues.apache.org/jira/browse/HIVE-13876
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats, Vectorization
>Affects Versions: 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Critical
> Attachments: HIVE-13876.1.patch
>
>
> High CPU usage due to exception handling code.
> {code}
>  TezTaskRunner [RUNNABLE] [DAEMON]
> java.lang.Throwable.fillInStackTrace(int) Throwable.java (native)
> java.lang.Throwable.fillInStackTrace() Throwable.java:783
> java.lang.Throwable.(String) Throwable.java:265
> java.lang.Exception.(String) Exception.java:66
> java.lang.RuntimeException.(String) RuntimeException.java:62
> java.lang.IllegalArgumentException.(String) 
> IllegalArgumentException.java:52
> java.lang.NumberFormatException.(String) NumberFormatException.java:55
> sun.misc.FloatingDecimal.readJavaFormatString(String) 
> FloatingDecimal.java:1842
> sun.misc.FloatingDecimal.parseFloat(String) FloatingDecimal.java:122
> java.lang.Float.parseFloat(String) Float.java:451
> org.apache.hadoop.hive.serde2.lazy.fast.LazySimpleDeserializeRead.readCheckNull()
>  LazySimpleDeserializeRead.java:309
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorizedRowBatch,
>  int, int) VectorDeserializeRow.java:346
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorizedRowBatch,
>  int) VectorDeserializeRow.java:659
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(Writable) 
> VectorMapOperator.java:814
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(Object) 
> MapRecordSource.java:86
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord() 
> MapRecordSource.java:70
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run() 
> MapRecordProcessor.java:361
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(Map,
>  Map) TezProcessor.java:172
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(Map, Map) 
> TezProcessor.java:160
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run() 
> LogicalIOProcessorRuntimeTask.java:370
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run() 
> TaskRunner2Callable.java:73
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run() 
> TaskRunner2Callable.java:61
> java.security.AccessController.doPrivileged(PrivilegedExceptionAction, 
> AccessControlContext) AccessController.java (native)
> javax.security.auth.Subject.doAs(Subject, PrivilegedExceptionAction) 
> Subject.java:422
> org.apache.hadoop.security.UserGroupInformation.doAs(PrivilegedExceptionAction)
>  UserGroupInformation.java:1657
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() 
> TaskRunner2Callable.java:61
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() 
> TaskRunner2Callable.java:37
> org.apache.tez.common.CallableWithNdc.call() CallableWithNdc.java:36
> java.util.concurrent.FutureTask.run() FutureTask.java:266
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) 
> ThreadPoolExecutor.java:1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run() 
> ThreadPoolExecutor.java:617
> java.lang.Thread.run() Thread.java:745
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)