[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-30 Thread Hive QA (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Hive QA commented on  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 
Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12850081/HIVE-14573.2.patch 
SUCCESS: +1 due to 1 test(s) being added or modified. 
ERROR: -1 due to 3 failed/errored test(s), 11011 tests executed Failed tests: 

 
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=223)
 

 
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3270/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3270/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3270/ 
Messages: 

 
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
 

 
This message is automatically generated. 
ATTACHMENT ID: 12850081 - PreCommit-HIVE-Build 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-30 Thread Teddy Choi (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Teddy Choi updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Hive /  HIVE-14573 
 
 
 
  Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 
The attached patch file name was wrong. I corrected it. 
 
 
 
 
 
 
 
 
 

Change By:
 
 Teddy Choi 
 
 
 

Attachment:
 
 HIVE-14573.2.patch 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-30 Thread Hive QA (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Hive QA commented on  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 
Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12849962/HIVE-15743.2.patch 
SUCCESS: +1 due to 1 test(s) being added or modified. 
ERROR: -1 due to 4 failed/errored test(s), 10996 tests executed Failed tests: 

 
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver (batchId=108)
 

 
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3261/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3261/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3261/ 
Messages: 

 
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
 

 
This message is automatically generated. 
ATTACHMENT ID: 12849962 - PreCommit-HIVE-Build 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
   

[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-30 Thread Teddy Choi (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Teddy Choi commented on  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 
I added more unit tests, and found a bug. It makes ArrayIndexOutOfBoundsExceptions when its pattern is an empty string. I fixed it. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-30 Thread Teddy Choi (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Teddy Choi updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Hive /  HIVE-14573 
 
 
 
  Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 

Change By:
 
 Teddy Choi 
 
 
 

Attachment:
 
 HIVE-15743.2.patch 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-29 Thread Gopal V (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Gopal V commented on  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 
Left a couple of comments on RB about tests, overall looks good. 
+1 tests/comments pending. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-29 Thread Hive QA (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Hive QA commented on  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 
Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12849908/HIVE-15743.1.patch 
ERROR: -1 due to no test(s) being added or modified. 
ERROR: -1 due to 6 failed/errored test(s), 11003 tests executed Failed tests: 

 
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_char_simple] (batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_varchar_simple] (batchId=153)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=93)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223)
 

 
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3251/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3251/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3251/ 
Messages: 

 
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
 

 
This message is automatically generated. 
ATTACHMENT ID: 12849908 - PreCommit-HIVE-Build 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
   

[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-29 Thread Teddy Choi (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Teddy Choi updated  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Hive /  HIVE-14573 
 
 
 
  Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 

Change By:
 
 Teddy Choi 
 
 
 

Status:
 
 Open Patch Available 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-29 Thread Teddy Choi (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Teddy Choi edited a comment on  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 I made an initial implementation. It copied [~gopalv]'s code. It works similar to Pattern and Matcher in JDK. StringExpr::compile() returns a Finder object. A Finder object uses Boyer Moore Horspool algorithm to find its pattern.I tried to combine compile and find into one, but it made performance loss. Then I tried to use ThreadLocal or synchronized blocks to implement in StringExpr: : find(), and both of them made performance loss, too. Separating compilation from finding was the most performant way. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-29 Thread Teddy Choi (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Teddy Choi edited a comment on  HIVE-14573 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 I made an initial implementation. It copied [~gopalv]'s code. It works similar to Pattern and Matcher in JDK. StringExpr::compile() returns a Finder object. A Finder object uses Boyer Moore Horspool algorithm to find its pattern.I tried to combine compile and find into one, but it made performance loss. Then I tried to use ThreadLocal or synchronized blocks to  combine them  implement in StringExpr:find() , and both of them made performance loss, too. Separating compilation from finding was the most performant way. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (HIVE-14573) Vectorization: Implement StringExpr::find()

2017-01-29 Thread Teddy Choi (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Teddy Choi updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Hive /  HIVE-14573 
 
 
 
  Vectorization: Implement StringExpr::find()   
 
 
 
 
 
 
 
 
 
 
I made an initial implementation. It copied Gopal V's code. It works similar to Pattern and Matcher in JDK. StringExpr::compile() returns a Finder object. A Finder object uses Boyer Moore Horspool algorithm to find its pattern. 
I tried to combine compile and find into one, but it made performance loss. Then I tried to use ThreadLocal or synchronized blocks to combine them, and both of them made performance loss, too. Separating compilation from finding was the most performant way. 
 
 
 
 
 
 
 
 
 

Change By:
 
 Teddy Choi 
 
 
 

Attachment:
 
 HIVE-15743.1.patch 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)