On Mon, Dec 10, 2012 at 8:00 PM, sebb <[email protected]> wrote: > On 8 December 2012 22:10, Philippe Mouawad <[email protected]> > wrote: > > Hello sebb, > > > > I will put it in some utility class, which one would be the best in your > > opinion ? > > > > I didn't test yet with Java 6 but Java 6 class has a reset method > although > > It has an text zone that contains matched test . > > > > But regarding the replacement you are talking about, it seems to me it's > > not that transparent as reading javadocs: > > > > - > http://docs.oracle.com/javase/6/docs/api/java/util/regex/Pattern.html > > - > > > http://jakarta.apache.org/oro/api/org/apache/oro/text/regex/Perl5Matcher.html > > > > Java class does not seem to fully support PERL5 syntax no ? so it could > > break existing test plan. > > AFAIK, ORO also does not support the full Perl5 syntax either. > > So it would be a question of checking whether the ORO features [1] [2] > are supported by Java. > > [1] > http://jakarta.apache.org/oro/api/org/apache/oro/text/regex/package-summary.html > [2] > http://www.savarese.org/oro/docs/OROMatcher/Syntax.html#Perl5Expressions > > One obvious omission from Java 6 is embedded comments. > However, they aren't essential. > > > So if we switch it seems to me we should have a configuration of regexp > > engine. > > I suppose that might be necessary initially, but ideally we would drop > support for ORO (in a later release) if Java 6 is compatible. > If Java 6 is not fully compatible, then it depends on how likely the > features are to have been used, whether it's possible to detect that > it has been used (so we could potentially report a suitable error) and > whether there is a work-round > > Java 6 has features not supported by ORO, and at least some of these > have been used by end-users. > > Ok
> > > > Anyway, I made a performance comparison on a test plan calling: > > > > - 2 samples that generate pages of 280 and 200 Ko, gzipped > respectively > > to 1714 and 1475 bytes > > - 200 threads > > - 3 s pause between each sample > > - 2 assertions > > - 3 regexp > > > > Results are interesting: > > > > - Memory and GC behaviours are greatly improved: > > - much less CPU consumed by GC activity 3 to 15% with 2.8, 2 to 8% > > with 2.9 > > - A Full GC drops memory to 100 Mb while with 2.8, it drops only to > > 200 Mb > > Sorry, I don't follow - what is being compared with what? > > I compared JMeter 2.8 with trunk version before the commits where I cached responseAsString on the test plan I commited called PerformanceTestPlanMemoryThread.jmx (which requires memory.jsp to be put in webapps/examples/jsp folder of tomcat) > Plan is maybe a bit agressive regarding page sizes. 200 Kb. > > > > Maybe we should create some kind of Base Test Plan. > > > > I think there is another potential improvement we could make which would > > be to cache in SampleResult#getResponseDataAsString the result of > > conversion. > > > > As if we have 3 regexp String will be decoded 3 times (CPU) and filled 3 > > times (memory). > > Not sure it uses extra memory - surely the decoded string will be > dropped (and made eligible for GC) once used? > > No in this case it consumes CPU. > > > > What do you think about it ? > > OK, but remember that it will affect the total data stored. > Yes but with last commits mixing cached responseAsString + cleanAfter sample I have memory close to what it was before caching and a neat CPU decrease as byte to String decoding was called 4 times instead of one now. > > > > > Regards > > > > Philippe > > > > On Sat, Dec 8, 2012 at 4:40 PM, sebb <[email protected]> wrote: > > > >> On 8 December 2012 13:03, <[email protected]> wrote: > >> > Author: pmouawad > >> > Date: Sat Dec 8 13:03:30 2012 > >> > New Revision: 1418666 > >> > > >> > URL: http://svn.apache.org/viewvc?rev=1418666&view=rev > >> > Log: > >> > Bug 54268 - Improve memory usage > >> > > >> > Bugzilla Id: 54268 > >> > > >> > Modified: > >> > > >> > jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java > >> > > >> > Modified: > >> > jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java > >> > URL: > >> > http://svn.apache.org/viewvc/jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java?rev=1418666&r1=1418665&r2=1418666&view=diff > >> > > >> > ============================================================================== > >> > --- > >> > jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java > >> (original) > >> > +++ > >> > jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java > >> Sat Dec 8 13:03:30 2012 > >> > @@ -110,11 +110,11 @@ public class RegexExtractor extends Abst > >> > if (defaultValue.length() > 0){// Only replace default if it > is > >> provided > >> > vars.put(refName, defaultValue); > >> > } > >> > - > >> > - > >> > + Perl5Matcher matcher = JMeterUtils.getMatcher(); > >> > String regex = getRegex(); > >> > + Pattern pattern = > >> JMeterUtils.getPatternCache().getPattern(regex, > >> Perl5Compiler.READ_ONLY_MASK); > >> > try { > >> > - List<MatchResult> matches = processMatches(regex, > >> previousResult, matchNumber, vars); > >> > + List<MatchResult> matches = processMatches(pattern, > regex, > >> previousResult, matchNumber, vars); > >> > int prevCount = 0; > >> > String prevString = vars.get(refName + REF_MATCH_NR); > >> > if (prevString != null) { > >> > @@ -162,6 +162,22 @@ public class RegexExtractor extends Abst > >> > } > >> > } catch (MalformedCachePatternException e) { > >> > log.warn("Error in pattern: " + regex); > >> > + } finally { > >> > + clearMatcherMemory(matcher, pattern); > >> > + } > >> > + } > >> > + > >> > + /** > >> > + * Hack to make matcher clean the two internal buffers it keeps > in > >> memory which size is equivalent to > >> > + * the unzipped page size > >> > + * @param matcher {@link Perl5Matcher} > >> > + * @param pattern Pattern > >> > + */ > >> > + private final void clearMatcherMemory(Perl5Matcher matcher, > Pattern > >> pattern) { > >> > + try { > >> > + matcher.matches("", pattern); // $NON-NLS-1$ > >> > + } catch (Exception e) { > >> > + // NOOP > >> > } > >> > >> Perhaps needs to be a utility method? > >> May be useful from other classes. > >> > >> [However, if we end up dropping ORO it will become irrelevant.] > >> > >> Does the Java regex processor behave in the same way? > >> Or is it better at tidying up? > >> > >> > } > >> > > >> > @@ -180,13 +196,12 @@ public class RegexExtractor extends Abst > >> > return inputString; > >> > } > >> > > >> > - private List<MatchResult> processMatches(String regex, > SampleResult > >> result, int matchNumber, JMeterVariables vars) { > >> > + private List<MatchResult> processMatches(Pattern pattern, String > >> regex, SampleResult result, int matchNumber, JMeterVariables vars) { > >> > if (log.isDebugEnabled()) { > >> > log.debug("Regex = " + regex); > >> > } > >> > > >> > Perl5Matcher matcher = JMeterUtils.getMatcher(); > >> > - Pattern pattern = > >> JMeterUtils.getPatternCache().getPattern(regex, > >> Perl5Compiler.READ_ONLY_MASK); > >> > List<MatchResult> matches = new ArrayList<MatchResult>(); > >> > int found = 0; > >> > > >> > > >> > > >> > > > > > > > > -- > > Cordialement. > > Philippe Mouawad. > -- Cordialement. Philippe Mouawad.
