[
https://issues.apache.org/jira/browse/NIFI-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833089#comment-15833089
]
Michael Moser commented on NIFI-3350:
-------------------------------------
Hello [~jameswing] I definitely appreciate your review!
It's the amount of time between these two log messages where documentation
extraction happens. This is the time I measured.
2017-01-21 11:26:54,410 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading
process took 23,239222261 nanoseconds.
2017-01-21 11:27:09,657 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded
NAR file:
D:\dev\nifi-master\.\work\nar\extensions\nifi-jetty-bundle-1.2.0-SNAPSHOT.nar-unpacked
as class loader
org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-jetty-bundle-1.2.0-SNAPSHOT.nar-unpacked]
On my Windows 10 box, I cleared out all 4 repositories and work directories and
started NiFi. The results were roughly the same on each run and looked like
this:
master branch ----------
2017-01-21 11:26:30,956 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2017-01-21 11:26:31,171 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding
58 NAR files with all processors... It can take few minutes.
2017-01-21 11:26:54,410 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading
process took 23,239222261 nanoseconds.
*** 15.2 seconds elapse before this next message
2017-01-21 11:27:09,657 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded
NAR file:
D:\dev\nifi-master\.\work\nar\extensions\nifi-jetty-bundle-1.2.0-SNAPSHOT.nar-unpacked
as class loader
org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-jetty-bundle-1.2.0-SNAPSHOT.nar-unpacked]
2017-01-21 11:27:24,943 INFO [main] org.apache.nifi.NiFi Controller
initialization took 12,847245943 nanoseconds.
*** 54.0 seconds total startup time
pr-1416 ----------
2017-01-21 12:01:32,996 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2017-01-21 12:01:33,179 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding
58 NAR files with all processors...
2017-01-21 12:01:56,727 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading
process took 23,549269902 nanoseconds.
*** 0.5 seconds elapse before this next message
2017-01-21 12:01:57,225 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded
NAR file:
D:\dev\nifi-1.2.0-SNAPSHOT\.\work\nar\extensions\nifi-jetty-bundle-1.2.0-SNAPSHOT.nar-unpacked
as class loader
org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-jetty-bundle-1.2.0-SNAPSHOT.nar-unpacked]
2017-01-21 12:02:12,711 INFO [main] org.apache.nifi.NiFi Controller
initialization took 12,956934776 nanoseconds.
*** 39.7 seconds total startup time
The PR shaved more than 14 seconds from a 54 second NiFi startup. I think I
could have made documentation extraction even faster with more code, but I
think I had already hit a point of diminishing returns. I did modify the "NAR
loading process took X nanoseconds" message because it should no longer take a
few minutes.
> Reduce NiFi startup time by streamlining documentation extraction
> -----------------------------------------------------------------
>
> Key: NIFI-3350
> URL: https://issues.apache.org/jira/browse/NIFI-3350
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 0.7.1, 1.1.1
> Reporter: Michael Moser
> Assignee: Michael Moser
> Attachments: nifi-3350-doc-optimization-results-1.png
>
>
> On NiFi startup, after seeing the new log message added in NIFI-3164, I
> noticed that generating documentation of all components takes a long time.
> After reading the NarUnpacker code, it appears to scan every file in every
> jar for NiFi component documentation. I think there are some optimizations
> we can do here which should reduce NiFi startup time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
