[
https://issues.apache.org/jira/browse/PIG-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756673#action_12756673
]
Thejas M Nair commented on PIG-965:
-----------------------------------
Hive like clause implementation is here -
http://svn.apache.org/viewvc/hadoop/hive/trunk/ql/src/java/org/apache/hadoop
/hive/ql/udf/UDFLike.java?revision=802066&view=markup
I ran simple tests with a simple java program to see the impact of these
optimizations. Optimization 1 reduces runtime to 1/2, optimization 2 reduces
runtime to 1/4 .
{code}
int matches =0;
int tot = 0;
String prefix = "123";
Pattern p = Pattern.compile("123.*");
while((str = in.readLine()) != null ){
//without proposed optimizations
//test setups 1 and 2 took 9secs, 126 secs respectively
// if(str.matches("123.*"))
// matches++;
// with optimization 1
// test sestups 1, 2 took 4, 57 secs respectively
// if((p.matcher(str).matches()))
// matches++;
// with optimization 1
// test sestups 1, 2 took 2.5, 25 secs respectively
//takes 2.5, 25 secs
// int len = prefix.length();
// boolean matched = true;
// for(int i=0; i<len; i++){
// if(prefix.charAt(i) != str.charAt(i)){
// matched = false;
// break;
// }
// }
// if(matched)
// matches++;
tot++;
}
}
System.out.println("matches " + matches + " tot " + tot);
{code}
> PERFORMANCE: optimize common case in matches (PORegex)
> ------------------------------------------------------
>
> Key: PIG-965
> URL: https://issues.apache.org/jira/browse/PIG-965
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: Thejas M Nair
>
> Some frequently seen use cases of 'matches' comparison operator have follow
> properties -
> 1. The rhs is a constant string . eg "c1 matches 'abc%' "
> 2. Regexes such that look for matching prefix , suffix etc are very common.
> eg - "abc%', "%abc", '%abc%'
> To optimize for these common cases , PORegex.java can be changed to -
> 1. Compile the pattern (rhs of matches) re-use it if the pattern string has
> not changed.
> 2. Use string comparisons for simple common regexes (in 2 above).
> The implementation of Hive like clause uses similar optimizations.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
