[jira] [Commented] (HIVE-13204) Vectorization: Add ChainedCheckerFactory for LIKE
[
https://issues.apache.org/jira/browse/HIVE-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184019#comment-15184019
]
Gopal V commented on HIVE-13204:
The 2 failed Hbase tests had unrelated mini-cluster start issues
{code}
Caused by: java.io.FileNotFoundException: File
file:/tmp/hadoop-yarn/staging/history/done does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:376)
at
org.apache.hadoop.fs.DelegateToFileSystem.listStatus(DelegateToFileSystem.java:149)
at org.apache.hadoop.fs.ChecksumFs.listStatus(ChecksumFs.java:516)
at
org.apache.hadoop.fs.AbstractFileSystem$1.(AbstractFileSystem.java:870)
at
org.apache.hadoop.fs.AbstractFileSystem.listStatusIterator(AbstractFileSystem.java:868)
at org.apache.hadoop.fs.FileContext$21.next(FileContext.java:1445)
at org.apache.hadoop.fs.FileContext$21.next(FileContext.java:1440)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1440)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:456)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:444)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:439)
at
org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.findTimestampedDirectories(HistoryFileManager.java:762)
at
org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.initExisting(HistoryFileManager.java:665)
at
org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:96)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:142)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster$JobHistoryServerWrapper.serviceStart(MiniMRYarnCluster.java:210)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:80)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:187)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:175)
at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:167)
at
org.apache.hive.hcatalog.hbase.ManyMiniCluster.setupMRCluster(ManyMiniCluster.java:218)
at
org.apache.hive.hcatalog.hbase.ManyMiniCluster.start(ManyMiniCluster.java:104)
at
org.apache.hive.hcatalog.hbase.SkeletonHBaseTest$Context.start(SkeletonHBaseTest.java:186)
at
org.apache.hive.hcatalog.hbase.SkeletonHBaseTest.setup(SkeletonHBaseTest.java:96
{code}
> Vectorization: Add ChainedCheckerFactory for LIKE
> --
>
> Key: HIVE-13204
> URL: https://issues.apache.org/jira/browse/HIVE-13204
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
> Attachments: HIVE-13204.1.patch, HIVE-13204.2.patch,
> HIVE-13204.3.patch, HIVE-13204.4.patch, regex-cpu-like.png
>
>
> Currently, Vectorization runs through a UTF-8 decode to produce a String &
> then check for mildly complex patterns like "http://%.exe; using a Regex.
> Since this pattern doesn't need any backtracking patterns, using a full
> fledged Regex is too expensive.
> !regex-cpu-like.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13204) Vectorization: Add ChainedCheckerFactory for LIKE
[
https://issues.apache.org/jira/browse/HIVE-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183987#comment-15183987
]
Hive QA commented on HIVE-13204:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12791583/HIVE-13204.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 9785 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7187/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7187/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7187/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12791583 - PreCommit-HIVE-TRUNK-Build
> Vectorization: Add ChainedCheckerFactory for LIKE
> --
>
> Key: HIVE-13204
> URL: https://issues.apache.org/jira/browse/HIVE-13204
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
> Attachments: HIVE-13204.1.patch, HIVE-13204.2.patch,
> HIVE-13204.3.patch, HIVE-13204.4.patch, regex-cpu-like.png
>
>
> Currently, Vectorization runs through a UTF-8 decode to produce a String &
> then check for mildly complex patterns like "http://%.exe; using a Regex.
> Since this pattern doesn't need any backtracking patterns, using a full
> fledged Regex is too expensive.
> !regex-cpu-like.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13204) Vectorization: Add ChainedCheckerFactory for LIKE
[ https://issues.apache.org/jira/browse/HIVE-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183801#comment-15183801 ] Gopal V commented on HIVE-13204: The existing tests for UDFLike don't actually run because they're missing @Test annotations. Fixing those tests in the next pass. > Vectorization: Add ChainedCheckerFactory for LIKE > -- > > Key: HIVE-13204 > URL: https://issues.apache.org/jira/browse/HIVE-13204 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Gopal V > Attachments: HIVE-13204.1.patch, HIVE-13204.2.patch, > HIVE-13204.3.patch, regex-cpu-like.png > > > Currently, Vectorization runs through a UTF-8 decode to produce a String & > then check for mildly complex patterns like "http://%.exe; using a Regex. > Since this pattern doesn't need any backtracking patterns, using a full > fledged Regex is too expensive. > !regex-cpu-like.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13204) Vectorization: Add ChainedCheckerFactory for LIKE
[ https://issues.apache.org/jira/browse/HIVE-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181414#comment-15181414 ] Sergey Shelukhin commented on HIVE-13204: - +1 pending tests > Vectorization: Add ChainedCheckerFactory for LIKE > -- > > Key: HIVE-13204 > URL: https://issues.apache.org/jira/browse/HIVE-13204 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Gopal V > Attachments: HIVE-13204.1.patch, HIVE-13204.2.patch, > HIVE-13204.3.patch, regex-cpu-like.png > > > Currently, Vectorization runs through a UTF-8 decode to produce a String & > then check for mildly complex patterns like "http://%.exe; using a Regex. > Since this pattern doesn't need any backtracking patterns, using a full > fledged Regex is too expensive. > !regex-cpu-like.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13204) Vectorization: Add ChainedCheckerFactory for LIKE
[
https://issues.apache.org/jira/browse/HIVE-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179666#comment-15179666
]
Gopal V commented on HIVE-13204:
h2. With
{code}
Status: Running (Executing on YARN cluster with App id
application_1455662455106_2535)
--
VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
--
Map 1 .. llap SUCCEEDED27027000
0 0
Reducer 2 .. llap SUCCEEDED 30 3000
0 0
--
VERTICES: 02/02 [==>>] 100% ELAPSED TIME: 73.26 s
--
Status: DAG finished successfully in 73.26 seconds
{code}
h2. Without
{code}
Status: Running (Executing on YARN cluster with App id
application_1455662455106_2535)
--
VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
--
Map 1 .. llap SUCCEEDED27027000
0 0
Reducer 2 .. llap SUCCEEDED 30 3000
0 0
--
VERTICES: 02/02 [==>>] 100% ELAPSED TIME: 378.36 s
--
Status: DAG finished successfully in 378.36 seconds
{code}
> Vectorization: Add ChainedCheckerFactory for LIKE
> --
>
> Key: HIVE-13204
> URL: https://issues.apache.org/jira/browse/HIVE-13204
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Affects Versions: 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
> Attachments: HIVE-13204.1.patch, regex-cpu-like.png
>
>
> Currently, Vectorization runs through a UTF-8 decode to produce a String &
> then check for mildly complex patterns like "http://%.exe; using a Regex.
> Since this pattern doesn't need any backtracking patterns, using a full
> fledged Regex is too expensive.
> !regex-cpu-like.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
