[GitHub] [nifi] FrederikP opened a new pull request #3500: NIFI-6322: Reuse evaluator stored in CompiledExpression when creating a query.

[GitBox]

FrederikP opened a new pull request #3500: NIFI-6322: Reuse evaluator stored in 
CompiledExpression when creating a query.
URL: https://github.com/apache/nifi/pull/3500
 
 
   
   #### Description of PR
   
   This commit shows how performance can be improved in expression language 
evaluation with a minimal set of changes to the code. The root evaluator stored 
in CompiledExpression instances is reused when creating queries instead of 
creating evaluators over and over again using the tree.
   
   #### Performance comparison
   
   We ran a before/after test using a slightly changed version of the test case 
found in the expression language module and profiled it. We set the number of 
iteration to 1M to not take forever when the code is instrumented and we 
changed the expression to one where the effect is even clearer. The effect is 
noticeable with the original expression, too though. (75% less CPU time)
   
   Here is the test case with our changes:
   
   ```
   @Test
   public void test10MIterations() {
       final Map<String, String> attrs = new HashMap<>();
       attrs.put("a", "world");
   
       final StandardPreparedQuery prepared = (StandardPreparedQuery) 
Query.prepare("${literal('world'):equals(${a})}");
       final long start = System.nanoTime();
       for (int i = 0; i < 1000000; i++) {
           assertEquals("true", prepared.evaluateExpressions(attrs, null));
       }
       final long nanos = System.nanoTime() - start;
       System.out.println(TimeUnit.NANOSECONDS.toMillis(nanos));
   }
   ```
   
   Here is the performance when run on the latest master of nifi without my 
change:
   
![before](https://user-images.githubusercontent.com/483596/58564556-6e77e080-822d-11e9-8246-3bc501c110cb.png)
   
   This is after applying my changes:
   
![after](https://user-images.githubusercontent.com/483596/58564573-78014880-822d-11e9-8b46-22fee37db11e.png)
   
   As you can see it takes about 40 seconds less with my changes. CPU time 
decreases by 85%. It gets rid of the most impactful call in every iteration: 
Query.fromTree()
   
   We have a huge amount of expressions in our production flow that's why this 
issue is so noticable for us. 
   
   Please let me know if my fix makes sense or if I'm missing something. I 
tried to keep it as simple as possible to demonstrate the issue in a clear way, 
so you might require some further changes to the structure to merge this.
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
        in the commit message?
   
   - [x] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [x] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [x] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [x] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes? -> 
Performance verified via performance unit test as described.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services