Reviving this thread for a similar question. During the build, nifi-registry/nifi-registry-core/nifi-registry-web-ui attempts another wget command.
[ERROR] gyp http GET https://nodejs.org/download/release/v10.16.3/node-v10.16.3-headers.tar.gz For building nifi (previous to 1.14.0), we were able to resolve Node issues by pre-staging the required node version locally, and providing the following maven option on the command line: -DnodeDownloadRoot=file://{node-repo-directory} Within that repo directory, we have v10.16.3 subdirectory which contains node-v10.16.3-linux-x64.tar.gz (for example.) No problems there. However, this same methodology does not seem to work for the required node-v10.16.3-headers.tar.gz file. Is there a similar maven switch to use to locate the node headers file? Thanks, Mark On Fri, Jul 16, 2021 at 12:29 PM Mark Bean <[email protected]> wrote: > Thanks for the feedback Matt. I created a JIRA ticket to track this. > > I also created a separate ticket to create root level profiles to > exclude (or selectively choose) modules now part of nifi such as > minifi, nifi-registry, etc. > > Any debate or opinions can be entered in the tickets > > https://issues.apache.org/jira/browse/NIFI-8916 > https://issues.apache.org/jira/browse/NIFI-8917 > > Thanks, > Mark > > > On Fri, Jul 16, 2021 at 11:57 AM Matt Burgess <[email protected]> > wrote: > >> Mark, >> >> MiNiFi Java has been generating Windows artifacts for a while now >> (before it became part of NiFi), and there are quite a few users who >> deploy MiNiFi Java on Windows. In my experience, users deploying to >> small devices tend to prefer MiNiFi C++ as long as the target >> architecture is supported. >> >> That being said, as long as the Windows artifacts are generated on >> release and can be generated on-demand, I'm not opposed to making the >> profile inactive by default, but would like to get others' thoughts on >> this as well. We would want to update any READMEs, Dev guides, etc. if >> we disable by default any profiles that used to be enabled. >> >> Regards, >> Matt >> >> On Fri, Jul 16, 2021 at 11:46 AM Mark Bean <[email protected]> wrote: >> > >> > Thanks for the explanation Kevin. And, I hope the transition to the JAR >> > goes well. >> > >> > Meanwhile, I found another point in the build that is reaching out to an >> > external location. The minifi assembly attempts to download some Windows >> > executable (profile windows-service-assembly). This is active by >> default, >> > which all by itself seems an odd thing since I would expect minifi to >> run >> > (typically) on small devices - not Windows-based systems. >> > >> > I'd like to remove all external dependencies from the build - or at >> least >> > have a profile to exclude it (if not required) similar to >> -Pno-swagger-ui. >> > And/or, it would be useful to have profiles at the project root to >> exclude >> > any one of the additional modules which are not explicitly part of >> "nifi", >> > but are now included in the project (nifi-registry, minifi, toolkit, >> etc.) >> > >> > Thanks, >> > Mark >> > >> > On Fri, Jul 16, 2021 at 11:12 AM Kevin Doran <[email protected]> >> > wrote: >> > >> > > I just compared the contents of the swagger UI tarball from GitHub >> > > artifacts and the jar from maven central, and they definitely look >> like >> > > they both contain the same files we are rebuilding in NiFi Registry, >> so I >> > > have opened a ticket to migrate from using the download-maven-plugin >> > > pulling from GitHub artifacts to use the maven artifact: >> > > >> > > https://issues.apache.org/jira/browse/NIFIREG-456 < >> > > https://issues.apache.org/jira/browse/NIFIREG-456> >> > > >> > > Thanks again, Mark! >> > > >> > > > On Jul 16, 2021, at 10:15 AM, Kevin Doran <[email protected]> >> > > wrote: >> > > > >> > > > Good find! At a glance, I think this maven jar could work as a >> > > replacement. I’ll look into it. >> > > > >> > > > Regarding how we use the Swagger UI. When included, the NiFi >> Registry >> > > web server hosts the Swagger UI in addition to the Registry web app >> and >> > > REST API. >> > > > >> > > > I believe it is hosted at >> > > http(s)://<host>:<port>/nifi-registry-api/swagger/ui.html >> > > > >> > > > The Swagger UI is an alternative to the REST API documentation that >> is >> > > also included in the Registry self-hosted docs (or here [1]). One of >> the >> > > main benefits of the Swagger UI, is that in addition to documenting >> usage >> > > of the REST API, it offers interactivity. If you’ve never seen >> Swagger UI >> > > before, can see a demo of it here [2]. It is completely data driven >> by a >> > > swagger.json spec, so in Registry it dynamically generates our REST >> API >> > > endpoints. >> > > > >> > > > [1] https://nifi.apache.org/docs/nifi-registry-docs/index.html >> > > > [2] https://petstore.swagger.io/ >> > > > >> > > >> On Jul 16, 2021, at 8:49 AM, Mark Bean <[email protected]> >> wrote: >> > > >> >> > > >> Thanks for laying out some clear options Kevin. >> > > >> >> > > >> I haven't dug into the inner workings of this component, and don't >> yet >> > > >> understand why a tarball is necessary. I looked at your first >> option. >> > > While >> > > >> I didn't find a tarball, there does appear to be >> org.webjars.swagger-ui >> > > >> JAR component in common maven repositories. I'm not sure if this is >> > > >> applicable - or could be made to be applicable. >> > > >> >> > > >> https://mvnrepository.com/artifact/org.webjars/swagger-ui >> > > >> https://search.maven.org/artifact/org.webjars/swagger-ui >> > > >> >> > > >> As a more general question, can you explain what option uses the >> Swagger >> > > >> UI? What feature or capability is not available without this >> component >> > > >> being available at build time? >> > > >> >> > > >> Thanks, >> > > >> Mark >> > > >> >> > > >> On Thu, Jul 15, 2021 at 4:52 PM Kevin Doran < >> [email protected]> >> > > wrote: >> > > >> >> > > >>> It’s certainly worth discussing. The binary in question is ~6MB, >> which >> > > >>> while not huge, if changed over time (to pull in new versions) >> could >> > > lead >> > > >>> to undesirable repository sizes. It's also debatable that it is >> > > “necessary” >> > > >>> as the bundled Swagger UI is an optional piece of NiFi Registry, >> which >> > > >>> itself is an optional component. Still we could handle it more >> > > gracefully. >> > > >>> >> > > >>> Options I can think of: >> > > >>> - See if there is an alternative Nexus / maven repository >> location, >> > > that >> > > >>> hosts the Swagger UI in a Jar format or something we could use as >> a >> > > Maven >> > > >>> dependency. >> > > >>> - Include the tarball in the source repo as Mark suggested >> > > >>> - Add an option to provide the swagger tarball as a local file, eg >> > > build >> > > >>> property to location >> > > >>> - Do what we do now, but detect if the tarball is not reachable >> via the >> > > >>> network and gracefully continue the build without that piece >> > > >>> >> > > >>>> On Jul 15, 2021, at 3:23 PM, Mark Bean <[email protected]> >> wrote: >> > > >>>> >> > > >>>> Thanks. That profile is what I needed to get beyond that point. >> (I ran >> > > >>> into >> > > >>>> other dependency issues I still need to resolve, so I don't yet >> have a >> > > >>>> complete build to test.) >> > > >>>> >> > > >>>> A more general question: should the wget functionality really be >> part >> > > of >> > > >>> a >> > > >>>> build? It seems to me that if a necessary component or dependency >> > > can't >> > > >>> be >> > > >>>> obtained from the user-configured maven repository, then that >> resource >> > > >>>> should be included in the source for the project. Comments? >> > > >>>> >> > > >>>> Thanks, >> > > >>>> Mark >> > > >>>> >> > > >>>> On Thu, Jul 15, 2021 at 2:11 PM Kevin Doran < >> [email protected]> >> > > >>> wrote: >> > > >>>> >> > > >>>>> Hi Mark, >> > > >>>>> >> > > >>>>> I’m not sure the two errors are related, but for the first one >> you >> > > can >> > > >>>>> pass -Pno-swagger-ui to mvn when building: >> > > >>>>> >> > > >>>>> See: >> > > >>>>> >> > > >>> >> > > >> https://github.com/apache/nifi/blob/main/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml#L136 >> > > >>>>> < >> > > >>>>> >> > > >>> >> > > >> https://github.com/apache/nifi/blob/main/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml#L136 >> > > >>>>>> >> > > >>>>> >> > > >>>>> Kevin >> > > >>>>> >> > > >>>>>> On Jul 15, 2021, at 1:54 PM, Mark Bean <[email protected]> >> > > wrote: >> > > >>>>>> >> > > >>>>>> I'm trying to build rel/nifi-1.14.0 on a private, non-Internet >> > > >>> connected >> > > >>>>>> network. The nifi-registry portion is failing. >> > > >>>>>> >> > > >>>>>> [ERROR] Failed to execute goal >> > > >>>>>> >> > > com.googlecode.maven-download-plugin:download-maven-plugin:1.2.1:wget >> > > >>>>>> (download-swagger-ui) on project nifi-registry-web-api: IO >> Error: >> > > Error >> > > >>>>>> while >> > > >>>>>> expalnding >> > > >>>>> >> > > >>> >> > > >> ~/nifi/nifi-registry/nifi-registry-core/nifi-registry-web-api/target/v3.12.0.tar.gz: >> > > >>>>>> Not in GZIP format >> > > >>>>>> [ERROR] Failed to execute goal >> > > >>>>>> com.github.eirslett:frontend-maven-plugin:1.5:npm >> (npm-install) on >> > > >>>>> project >> > > >>>>>> nifi-registry-web-ui: Failed to run task: 'npm --silent ci' >> failed. >> > > >>>>>> org.apache.commons.exec.ExecuteException: Process exited with >> an >> > > >>> error: 1 >> > > >>>>>> (Exit value: 1) >> > > >>>>>> >> > > >>>>>> The two errors may be related (?) Either way, it appears to me >> that >> > > the >> > > >>>>>> build is attempting to use wget to reach out to an Internet >> > > location to >> > > >>>>>> download the v3.12.0.tar.gz. >> > > >>>>>> >> > > >>>>>> Can someone confirm that is the problem? Is there a way around >> this? >> > > >>> Can >> > > >>>>>> this file be provided from a reachable/local source location? >> > > >>>>>> >> > > >>>>>> Thanks, >> > > >>>>>> Mark >> > > >>>>> >> > > >>>>> >> > > >>> >> > > >>> >> > > > >> > > >> > > >> >
