markap14 commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1164391462
##########
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-utils/src/main/java/org/apache/nifi/nar/StandardExtensionDiscoveringManager.java:
##########
@@ -169,6 +181,75 @@ public void discoverExtensions(final Set<Bundle>
narBundles, final boolean logDe
}
}
+ public void setPythonBridge(final PythonBridge pythonBridge) {
+ this.pythonBridge = pythonBridge;
+ }
+
+ @Override
+ public void discoverPythonExtensions(final Bundle pythonBundle) {
+ logger.info("Scanning to discover which Python extensions are
available and importing any necessary dependencies. This may take a few
minutes. See python logs for more details.");
+ final long start = System.currentTimeMillis();
+ pythonBridge.discoverExtensions();
+
+
bundleCoordinateBundleLookup.putIfAbsent(pythonBundle.getBundleDetails().getCoordinate(),
pythonBundle);
+
+ final Set<ExtensionDefinition> processorDefinitions =
definitionMap.get(Processor.class);
+ final List<PythonProcessorDetails> pythonProcessorDetails =
pythonBridge.getProcessorTypes();
+
+ int processorsFound = 0;
+ for (final PythonProcessorDetails details : pythonProcessorDetails) {
+ final BundleDetails bundleDetails =
createBundleDetailsWithOverriddenVersion(pythonBundle.getBundleDetails(),
details.getProcessorVersion());
+ final Bundle bundle = new Bundle(bundleDetails,
pythonBundle.getClassLoader());
+
+ // TODO: This is a workaround because the UI has a bug that causes
it not to work properly if the type doesn't have a '.' in it
+ final String className = "python." + details.getProcessorType();
+ final ExtensionDefinition extensionDefinition = new
ExtensionDefinition.Builder()
+ .implementationClassName(className)
+ .runtime(ExtensionRuntime.PYTHON)
+ .bundle(bundle)
+ .extensionType(Processor.class)
+ .description(details.getCapabilityDescription())
+ .tags(details.getTags())
+ .version(details.getProcessorVersion())
+ .build();
+
+ final boolean added =
processorDefinitions.add(extensionDefinition);
+ if (added) {
+ processorsFound++;
+ final List<Bundle> bundlesForClass =
classNameBundleLookup.computeIfAbsent(className, key -> new ArrayList<>());
+ bundlesForClass.add(bundle);
+
bundleCoordinateBundleLookup.putIfAbsent(bundleDetails.getCoordinate(), bundle);
+ logger.info("Discovered Python Processor {}",
details.getProcessorType());
+ } else {
+ logger.debug("Python Processor {} is already known",
details.getProcessorType());
+ }
+ }
+
+ logger.info("Discovered {} new/updated Python Processors in {}
millis", processorsFound, System.currentTimeMillis() - start);
Review Comment:
👍 I will change it to log at debug level if it doesn't find anything. This
allows us to ensure that the process is happening. But if at least one
processor is found/updated I'll log at info level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
