[
https://issues.apache.org/jira/browse/NIFI-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833231#comment-15833231
]
James Wing commented on NIFI-3350:
----------------------------------
[~mosermw], thank you for explaining, I misunderstood. My measurements above
were completely wrong. I was measuring from the log entry "NAR loading process
took 63402154252 nanoseconds", which I believed we were trying to change. Now
that I understand we are optimizing the step after that, I am (hopefully)
measuring the impact more clearly.
I made a two slightly modified copies of PR 1416 to log timing of both what the
master branch does now, and what the optimization would do. The results of my
not-so-scientific testing are in a [Google
Spreadsheet|https://docs.google.com/spreadsheets/d/1cokVMNb7opj221cfi99jd2_dEeKfDZl_X5QRcKvCd3M/edit?usp=sharing],
for the very curious. The short, short version is that I replicated your time
savings:
* Out of 1,285 JAR files, we now process documentation for only 53, skipping
1,232.
* Documentation processing time dropped from about 22 seconds to less than 1
second (on my containerized test NiFi).
* Generated documentation was identical, as tested by a recursive diff, with
the exception of changes to the Expression Language Guide from
[3bb876|https://github.com/apache/nifi/commit/3bb876eb835499845e66848e815cd166b683232c#diff-daac74ec3b89e26d99806d7a90254fe3],
commited after PR 1416 was branched.
I'm still doing some code review, but I wanted to report that I'm now testing
the right thing.
> Reduce NiFi startup time by streamlining documentation extraction
> -----------------------------------------------------------------
>
> Key: NIFI-3350
> URL: https://issues.apache.org/jira/browse/NIFI-3350
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 0.7.1, 1.1.1
> Reporter: Michael Moser
> Assignee: Michael Moser
> Attachments: nifi-3350-doc-optimization-results-1.png
>
>
> On NiFi startup, after seeing the new log message added in NIFI-3164, I
> noticed that generating documentation of all components takes a long time.
> After reading the NarUnpacker code, it appears to scan every file in every
> jar for NiFi component documentation. I think there are some optimizations
> we can do here which should reduce NiFi startup time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
