Thanks for the feedback Matt. I created a JIRA ticket to track this. I also created a separate ticket to create root level profiles to exclude (or selectively choose) modules now part of nifi such as minifi, nifi-registry, etc.
Any debate or opinions can be entered in the tickets https://issues.apache.org/jira/browse/NIFI-8916 https://issues.apache.org/jira/browse/NIFI-8917 Thanks, Mark On Fri, Jul 16, 2021 at 11:57 AM Matt Burgess <[email protected]> wrote: > Mark, > > MiNiFi Java has been generating Windows artifacts for a while now > (before it became part of NiFi), and there are quite a few users who > deploy MiNiFi Java on Windows. In my experience, users deploying to > small devices tend to prefer MiNiFi C++ as long as the target > architecture is supported. > > That being said, as long as the Windows artifacts are generated on > release and can be generated on-demand, I'm not opposed to making the > profile inactive by default, but would like to get others' thoughts on > this as well. We would want to update any READMEs, Dev guides, etc. if > we disable by default any profiles that used to be enabled. > > Regards, > Matt > > On Fri, Jul 16, 2021 at 11:46 AM Mark Bean <[email protected]> wrote: > > > > Thanks for the explanation Kevin. And, I hope the transition to the JAR > > goes well. > > > > Meanwhile, I found another point in the build that is reaching out to an > > external location. The minifi assembly attempts to download some Windows > > executable (profile windows-service-assembly). This is active by default, > > which all by itself seems an odd thing since I would expect minifi to run > > (typically) on small devices - not Windows-based systems. > > > > I'd like to remove all external dependencies from the build - or at least > > have a profile to exclude it (if not required) similar to > -Pno-swagger-ui. > > And/or, it would be useful to have profiles at the project root to > exclude > > any one of the additional modules which are not explicitly part of > "nifi", > > but are now included in the project (nifi-registry, minifi, toolkit, > etc.) > > > > Thanks, > > Mark > > > > On Fri, Jul 16, 2021 at 11:12 AM Kevin Doran <[email protected]> > > wrote: > > > > > I just compared the contents of the swagger UI tarball from GitHub > > > artifacts and the jar from maven central, and they definitely look like > > > they both contain the same files we are rebuilding in NiFi Registry, > so I > > > have opened a ticket to migrate from using the download-maven-plugin > > > pulling from GitHub artifacts to use the maven artifact: > > > > > > https://issues.apache.org/jira/browse/NIFIREG-456 < > > > https://issues.apache.org/jira/browse/NIFIREG-456> > > > > > > Thanks again, Mark! > > > > > > > On Jul 16, 2021, at 10:15 AM, Kevin Doran <[email protected]> > > > wrote: > > > > > > > > Good find! At a glance, I think this maven jar could work as a > > > replacement. I’ll look into it. > > > > > > > > Regarding how we use the Swagger UI. When included, the NiFi Registry > > > web server hosts the Swagger UI in addition to the Registry web app and > > > REST API. > > > > > > > > I believe it is hosted at > > > http(s)://<host>:<port>/nifi-registry-api/swagger/ui.html > > > > > > > > The Swagger UI is an alternative to the REST API documentation that > is > > > also included in the Registry self-hosted docs (or here [1]). One of > the > > > main benefits of the Swagger UI, is that in addition to documenting > usage > > > of the REST API, it offers interactivity. If you’ve never seen Swagger > UI > > > before, can see a demo of it here [2]. It is completely data driven by > a > > > swagger.json spec, so in Registry it dynamically generates our REST API > > > endpoints. > > > > > > > > [1] https://nifi.apache.org/docs/nifi-registry-docs/index.html > > > > [2] https://petstore.swagger.io/ > > > > > > > >> On Jul 16, 2021, at 8:49 AM, Mark Bean <[email protected]> > wrote: > > > >> > > > >> Thanks for laying out some clear options Kevin. > > > >> > > > >> I haven't dug into the inner workings of this component, and don't > yet > > > >> understand why a tarball is necessary. I looked at your first > option. > > > While > > > >> I didn't find a tarball, there does appear to be > org.webjars.swagger-ui > > > >> JAR component in common maven repositories. I'm not sure if this is > > > >> applicable - or could be made to be applicable. > > > >> > > > >> https://mvnrepository.com/artifact/org.webjars/swagger-ui > > > >> https://search.maven.org/artifact/org.webjars/swagger-ui > > > >> > > > >> As a more general question, can you explain what option uses the > Swagger > > > >> UI? What feature or capability is not available without this > component > > > >> being available at build time? > > > >> > > > >> Thanks, > > > >> Mark > > > >> > > > >> On Thu, Jul 15, 2021 at 4:52 PM Kevin Doran < > [email protected]> > > > wrote: > > > >> > > > >>> It’s certainly worth discussing. The binary in question is ~6MB, > which > > > >>> while not huge, if changed over time (to pull in new versions) > could > > > lead > > > >>> to undesirable repository sizes. It's also debatable that it is > > > “necessary” > > > >>> as the bundled Swagger UI is an optional piece of NiFi Registry, > which > > > >>> itself is an optional component. Still we could handle it more > > > gracefully. > > > >>> > > > >>> Options I can think of: > > > >>> - See if there is an alternative Nexus / maven repository location, > > > that > > > >>> hosts the Swagger UI in a Jar format or something we could use as a > > > Maven > > > >>> dependency. > > > >>> - Include the tarball in the source repo as Mark suggested > > > >>> - Add an option to provide the swagger tarball as a local file, eg > > > build > > > >>> property to location > > > >>> - Do what we do now, but detect if the tarball is not reachable > via the > > > >>> network and gracefully continue the build without that piece > > > >>> > > > >>>> On Jul 15, 2021, at 3:23 PM, Mark Bean <[email protected]> > wrote: > > > >>>> > > > >>>> Thanks. That profile is what I needed to get beyond that point. > (I ran > > > >>> into > > > >>>> other dependency issues I still need to resolve, so I don't yet > have a > > > >>>> complete build to test.) > > > >>>> > > > >>>> A more general question: should the wget functionality really be > part > > > of > > > >>> a > > > >>>> build? It seems to me that if a necessary component or dependency > > > can't > > > >>> be > > > >>>> obtained from the user-configured maven repository, then that > resource > > > >>>> should be included in the source for the project. Comments? > > > >>>> > > > >>>> Thanks, > > > >>>> Mark > > > >>>> > > > >>>> On Thu, Jul 15, 2021 at 2:11 PM Kevin Doran < > [email protected]> > > > >>> wrote: > > > >>>> > > > >>>>> Hi Mark, > > > >>>>> > > > >>>>> I’m not sure the two errors are related, but for the first one > you > > > can > > > >>>>> pass -Pno-swagger-ui to mvn when building: > > > >>>>> > > > >>>>> See: > > > >>>>> > > > >>> > > > > https://github.com/apache/nifi/blob/main/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml#L136 > > > >>>>> < > > > >>>>> > > > >>> > > > > https://github.com/apache/nifi/blob/main/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml#L136 > > > >>>>>> > > > >>>>> > > > >>>>> Kevin > > > >>>>> > > > >>>>>> On Jul 15, 2021, at 1:54 PM, Mark Bean <[email protected]> > > > wrote: > > > >>>>>> > > > >>>>>> I'm trying to build rel/nifi-1.14.0 on a private, non-Internet > > > >>> connected > > > >>>>>> network. The nifi-registry portion is failing. > > > >>>>>> > > > >>>>>> [ERROR] Failed to execute goal > > > >>>>>> > > > com.googlecode.maven-download-plugin:download-maven-plugin:1.2.1:wget > > > >>>>>> (download-swagger-ui) on project nifi-registry-web-api: IO > Error: > > > Error > > > >>>>>> while > > > >>>>>> expalnding > > > >>>>> > > > >>> > > > > ~/nifi/nifi-registry/nifi-registry-core/nifi-registry-web-api/target/v3.12.0.tar.gz: > > > >>>>>> Not in GZIP format > > > >>>>>> [ERROR] Failed to execute goal > > > >>>>>> com.github.eirslett:frontend-maven-plugin:1.5:npm (npm-install) > on > > > >>>>> project > > > >>>>>> nifi-registry-web-ui: Failed to run task: 'npm --silent ci' > failed. > > > >>>>>> org.apache.commons.exec.ExecuteException: Process exited with an > > > >>> error: 1 > > > >>>>>> (Exit value: 1) > > > >>>>>> > > > >>>>>> The two errors may be related (?) Either way, it appears to me > that > > > the > > > >>>>>> build is attempting to use wget to reach out to an Internet > > > location to > > > >>>>>> download the v3.12.0.tar.gz. > > > >>>>>> > > > >>>>>> Can someone confirm that is the problem? Is there a way around > this? > > > >>> Can > > > >>>>>> this file be provided from a reachable/local source location? > > > >>>>>> > > > >>>>>> Thanks, > > > >>>>>> Mark > > > >>>>> > > > >>>>> > > > >>> > > > >>> > > > > > > > > > > >
