Re: [PR] NIFI-12889: Retry Kerberos login on auth failure in HDFS processors [nifi]
Lehel44 commented on code in PR #8495:
URL: https://github.com/apache/nifi/pull/8495#discussion_r1554481562
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/GetHDFSSequenceFile.java:
##
@@ -108,6 +109,8 @@ protected void processBatchOfFiles(final List files,
final ProcessContext
if (!keepSourceFiles && !hdfs.delete(file, false)) {
logger.warn("Unable to delete path " + file.toString() + "
from HDFS. Will likely be picked up over and over...");
}
+} catch (final IOException e) {
Review Comment:
I think in the default exception handling the
```java
session.rollback();
context.yield();
```
was removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12889: Retry Kerberos login on auth failure in HDFS processors [nifi]
Lehel44 commented on code in PR #8495:
URL: https://github.com/apache/nifi/pull/8495#discussion_r1554480782
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java:
##
@@ -352,95 +355,95 @@ protected void processBatchOfFiles(final List
files, final ProcessContext
for (final Path file : files) {
-ugi.doAs(new PrivilegedAction() {
-@Override
-public Object run() {
-FlowFile flowFile = session.create(parentFlowFile);
-try {
-final String originalFilename = file.getName();
-final Path outputDirPath = getNormalizedPath(context,
OUTPUT_DIRECTORY, parentFlowFile);
-final Path newFile = new Path(outputDirPath,
originalFilename);
-final boolean destinationExists = hdfs.exists(newFile);
-// If destination file already exists, resolve that
-// based on processor configuration
-if (destinationExists) {
-switch (processorConfig.getConflictResolution()) {
-case REPLACE_RESOLUTION:
-// Remove destination file (newFile) to
replace
-if (hdfs.delete(newFile, false)) {
-getLogger().info("deleted {} in order
to replace with the contents of {}",
-new Object[]{newFile,
flowFile});
-}
-break;
-case IGNORE_RESOLUTION:
-session.transfer(flowFile, REL_SUCCESS);
-getLogger().info(
-"transferring {} to success
because file with same name already exists",
-new Object[]{flowFile});
-return null;
-case FAIL_RESOLUTION:
-
session.transfer(session.penalize(flowFile), REL_FAILURE);
-getLogger().warn(
-"penalizing {} and routing to
failure because file with same name already exists",
-new Object[]{flowFile});
-return null;
-default:
-break;
-}
+ugi.doAs((PrivilegedAction) () -> {
Review Comment:
I think the patch didn't apply correctly. The NullPointerException still
occurs. In the processBatchOfFiles method's catch should look like this:
```java
catch (final Throwable t) {
final Optional causeOptional =
findCause(t, GSSException.class, gsse -> GSSException.NO_CRED ==
gsse.getMajor());
if (causeOptional.isPresent()) {
throw new UncheckedIOException(new
IOException(causeOptional.get()));
}
getLogger().error("Failed to rename on HDFS due to {}",
new Object[]{t});
session.transfer(session.penalize(flowFile),
REL_FAILURE);
context.yield();
}
```
In case we roll back the session here in the for loop, the
NullPointerException appears, that's why we handle the exception outside of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12889: Retry Kerberos login on auth failure in HDFS processors [nifi]
Lehel44 commented on code in PR #8495:
URL: https://github.com/apache/nifi/pull/8495#discussion_r1554480782
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java:
##
@@ -352,95 +355,95 @@ protected void processBatchOfFiles(final List
files, final ProcessContext
for (final Path file : files) {
-ugi.doAs(new PrivilegedAction() {
-@Override
-public Object run() {
-FlowFile flowFile = session.create(parentFlowFile);
-try {
-final String originalFilename = file.getName();
-final Path outputDirPath = getNormalizedPath(context,
OUTPUT_DIRECTORY, parentFlowFile);
-final Path newFile = new Path(outputDirPath,
originalFilename);
-final boolean destinationExists = hdfs.exists(newFile);
-// If destination file already exists, resolve that
-// based on processor configuration
-if (destinationExists) {
-switch (processorConfig.getConflictResolution()) {
-case REPLACE_RESOLUTION:
-// Remove destination file (newFile) to
replace
-if (hdfs.delete(newFile, false)) {
-getLogger().info("deleted {} in order
to replace with the contents of {}",
-new Object[]{newFile,
flowFile});
-}
-break;
-case IGNORE_RESOLUTION:
-session.transfer(flowFile, REL_SUCCESS);
-getLogger().info(
-"transferring {} to success
because file with same name already exists",
-new Object[]{flowFile});
-return null;
-case FAIL_RESOLUTION:
-
session.transfer(session.penalize(flowFile), REL_FAILURE);
-getLogger().warn(
-"penalizing {} and routing to
failure because file with same name already exists",
-new Object[]{flowFile});
-return null;
-default:
-break;
-}
+ugi.doAs((PrivilegedAction) () -> {
Review Comment:
I think the patch didn't apply correctly. The NullPointerException still
occurs. In the processBatchOfFiles method's catch should look like this:
```java
catch (final Throwable t) {
final Optional causeOptional =
findCause(t, GSSException.class, gsse -> GSSException.NO_CRED ==
gsse.getMajor());
if (causeOptional.isPresent()) {
throw new UncheckedIOException(new
IOException(causeOptional.get()));
}
getLogger().error("Failed to rename on HDFS due to {}",
new Object[]{t});
session.transfer(session.penalize(flowFile),
REL_FAILURE);
context.yield();
}
```
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java:
##
@@ -352,95 +355,95 @@ protected void processBatchOfFiles(final List
files, final ProcessContext
for (final Path file : files) {
-ugi.doAs(new PrivilegedAction() {
-@Override
-public Object run() {
-FlowFile flowFile = session.create(parentFlowFile);
-try {
-final String originalFilename = file.getName();
-final Path outputDirPath = getNormalizedPath(context,
OUTPUT_DIRECTORY, parentFlowFile);
-final Path newFile = new Path(outputDirPath,
originalFilename);
-final boolean destinationExists = hdfs.exists(newFile);
-// If destination file already exists, resolve that
-// based on processor configuration
-if (destinationExists) {
-switch (processorConfig.getConflictResolution()) {
-case REPLACE_RESOLUTION:
-// Remove destination file (newFile) to
replace
-if (hdfs.delete(newFile, false)) {
-getLogger().info("deleted
Re: [PR] NIFI-12889: Retry Kerberos login on auth failure in HDFS processors [nifi]
Lehel44 commented on code in PR #8495:
URL: https://github.com/apache/nifi/pull/8495#discussion_r1554480782
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java:
##
@@ -352,95 +355,95 @@ protected void processBatchOfFiles(final List
files, final ProcessContext
for (final Path file : files) {
-ugi.doAs(new PrivilegedAction() {
-@Override
-public Object run() {
-FlowFile flowFile = session.create(parentFlowFile);
-try {
-final String originalFilename = file.getName();
-final Path outputDirPath = getNormalizedPath(context,
OUTPUT_DIRECTORY, parentFlowFile);
-final Path newFile = new Path(outputDirPath,
originalFilename);
-final boolean destinationExists = hdfs.exists(newFile);
-// If destination file already exists, resolve that
-// based on processor configuration
-if (destinationExists) {
-switch (processorConfig.getConflictResolution()) {
-case REPLACE_RESOLUTION:
-// Remove destination file (newFile) to
replace
-if (hdfs.delete(newFile, false)) {
-getLogger().info("deleted {} in order
to replace with the contents of {}",
-new Object[]{newFile,
flowFile});
-}
-break;
-case IGNORE_RESOLUTION:
-session.transfer(flowFile, REL_SUCCESS);
-getLogger().info(
-"transferring {} to success
because file with same name already exists",
-new Object[]{flowFile});
-return null;
-case FAIL_RESOLUTION:
-
session.transfer(session.penalize(flowFile), REL_FAILURE);
-getLogger().warn(
-"penalizing {} and routing to
failure because file with same name already exists",
-new Object[]{flowFile});
-return null;
-default:
-break;
-}
+ugi.doAs((PrivilegedAction) () -> {
Review Comment:
I think the patch didn't apply correctly. The NullPointerException still
occurs. In the processBatchOfFiles method's catch should look like this:
```java
catch (final Throwable t) {
final Optional causeOptional =
findCause(t, GSSException.class, gsse -> GSSException.NO_CRED ==
gsse.getMajor());
if (causeOptional.isPresent()) {
throw new UncheckedIOException(new
IOException(causeOptional.get()));
}
getLogger().error("Failed to rename on HDFS due to {}",
new Object[]{t});
session.transfer(session.penalize(flowFile),
REL_FAILURE);
context.yield();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-13005) Update Component State Dialog for Clustering
[ https://issues.apache.org/jira/browse/NIFI-13005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13005: --- Status: Patch Available (was: In Progress) > Update Component State Dialog for Clustering > > > Key: NIFI-13005 > URL: https://issues.apache.org/jira/browse/NIFI-13005 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When the component state dialog was introduced the clustering details were > not included. These need to be added. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13005: Cluster State in Component State Dialog [nifi]
mcgilman opened a new pull request, #8609: URL: https://github.com/apache/nifi/pull/8609 NIFI-13005: - Adding support for showing cluster state in the component state dialog. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13004) Remove include-grpc from Developer Guide
[ https://issues.apache.org/jira/browse/NIFI-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13004: Priority: Minor (was: Major) > Remove include-grpc from Developer Guide > > > Key: NIFI-13004 > URL: https://issues.apache.org/jira/browse/NIFI-13004 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M2 >Reporter: Mark Bean >Assignee: Mark Bean >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The `include-grpc` profile was removed in NIFI-12069. However, a reference to > it still remains in the Developer Guide. Remove the reference. And check for > other references. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13004) Remove include-grpc from Developer Guide
[ https://issues.apache.org/jira/browse/NIFI-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13004: Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Remove include-grpc from Developer Guide > > > Key: NIFI-13004 > URL: https://issues.apache.org/jira/browse/NIFI-13004 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M2 >Reporter: Mark Bean >Assignee: Mark Bean >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The `include-grpc` profile was removed in NIFI-12069. However, a reference to > it still remains in the Developer Guide. Remove the reference. And check for > other references. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13004) Remove include-grpc from Developer Guide
[ https://issues.apache.org/jira/browse/NIFI-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834443#comment-17834443 ] ASF subversion and git services commented on NIFI-13004: Commit 506ac835ad0111930c8c6df60ad5d0cf2ccc0ea4 in nifi's branch refs/heads/main from Mark Bean [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=506ac835ad ] NIFI-13004 Removed include-grpc profile from documentation This closes #8605 Signed-off-by: David Handermann > Remove include-grpc from Developer Guide > > > Key: NIFI-13004 > URL: https://issues.apache.org/jira/browse/NIFI-13004 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M2 >Reporter: Mark Bean >Assignee: Mark Bean >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The `include-grpc` profile was removed in NIFI-12069. However, a reference to > it still remains in the Developer Guide. Remove the reference. And check for > other references. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13004 remove include-grpc profile from developer and walkthrough… [nifi]
exceptionfactory commented on PR #8605: URL: https://github.com/apache/nifi/pull/8605#issuecomment-2040650258 This change is part of the generated documentation, which will be published as part of the next release process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-13004 remove include-grpc profile from developer and walkthrough… [nifi]
exceptionfactory closed pull request #8605: NIFI-13004 remove include-grpc profile from developer and walkthrough… URL: https://github.com/apache/nifi/pull/8605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12993 Add auto commit feature and add batch processing [nifi]
jrsteinebrey commented on PR #8597: URL: https://github.com/apache/nifi/pull/8597#issuecomment-2040638048 @dan-s1 I accidentally hit a button that requested yoiu rereview this PR. That is up to you. I intended to ask for the rereview from mattyb149 and he intends to look at it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13005) Update Component State Dialog for Clustering
Matt Gilman created NIFI-13005: -- Summary: Update Component State Dialog for Clustering Key: NIFI-13005 URL: https://issues.apache.org/jira/browse/NIFI-13005 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman When the component state dialog was introduced the clustering details were not included. These need to be added. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) ** Counter table has extra unnecessary can modify check (NIFI-12948) * Bulletin Board (NIFI-12560) * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) ** Clustering (NIFI-12807) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry (NIFI-12734) * Parameter Providers (NIFI-12622) ** Fetch parameters from provider, map to parameter context (dialog) - (NIFI-12665) * Cluster ** Node table (Disconnect/Connect/Load Balance/Etc) ** Status History - node specific values (NIFI-12848) * Flow Configuration History (NIFI-12754) ** ActionEntity.action should be optional (NIFI-12948) * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) ** Don't show users or groups in create/edit dialog is there are none (NIFI-12948) * Policies (NIFI-12548) ** Overridden policy Empty or Copy (NIFI-12679) ** Select Empty by default (NIFI-12948) * Help (NIFI-12795) * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) ** Clustering (NIFI-12807) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) ** Clustering (NIFI-13005) * Change Component Version * Consider PG permissions in Toolbox (NIFI-12683) * Handle linking to components that are not on the canvas * PG Version (NIFI-12963 & NIFI-12995) ** Start (NIFI-12963) ** Commit (NIFI-12963) ** Force Commit (NIFI-12963) ** Show changes (NIFI-12995) ** Revert changes (NIFI-12995) ** Change Flow version (NIFI-12995) ** Stop (NIFI-12963) * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs ** Disable and Configure * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Stop and Configure ** Open Custom UI (NIFI-12958) ** Property History ** Unable to re-add any removed Property (NIFI-12743) ** Shift-Enter new line when editing Property (NIFI-12743) * Property Verification * More Details (Processor, Controller Service, Reporting Task) * Download Flow * Create RPG (NIFI-12758) * Configure RPG (NIFI-12774) * RPG Remote Ports (NIFI-12778) * RPG Go To (NIFI-12759) * RPG Refresh (NIFI-12761) * Color * Move to Front * Copy/Paste * Add/Update Info Icons in dialogs throughout the application * Set viewport earlier when loading a Process Group (NIFI-12737) * Canvas global menu item should navigate user back to where they were on the canvas (NIFI-12737) * Better theme support (NIFI-12655) * Set up development/production environments files * Run unit tests are part of standard build (NIFI-12941) * Update all API calls to consider disconnect node confirmation (NIFI-13001) * Update API calls to use uiOnly flag (NIFI-12950) * Use polling interval from API * Load FlowConfiguration in guard (NIFI-12948) * Routing error handling * General API response error handling ** Management CS (NIFI-12663) ** Canvas CS (NIFI-12684) ** Remainder of Settings (NIFI-12723) ** Counters (NIFI-12723) ** Bulletins (NIFI-12723) ** Flow Designer ** Parameter Contexts (NIFI-12937) ** Parameter ** Provenance (NIFI-12767) ** Queue Listing (NIFI-12742) ** Summary (NIFI-12742) ** Users (NIFI-12742) ** Policies ** Status History * Introduce header in new pages to unify with canvas and offer better navigation. (NIFI-12597) * Theme docs, view flow file, and custom ui's * Prompt user to save Parameter Context when Edit form is dirty * Upgrade to Angular 17 (NIFI-12790) * Start/Stop processors, process
[jira] [Updated] (NIFI-12959) Support loading Python processors from NARs
[
https://issues.apache.org/jira/browse/NIFI-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12959:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Support loading Python processors from NARs
> ---
>
> Key: NIFI-12959
> URL: https://issues.apache.org/jira/browse/NIFI-12959
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently, third-party dependencies for Python Processors can be handled in
> two ways. Either they can be declared as dependencies in a Processor itself;
> or the Processor can be in a module where a {{requirements.txt}} dictates the
> requirements. These can be very helpful for developing Python based
> Processors.
> However, in production environments, it is not uncommon to see environments
> where {{pip}} is not installed. There is an inherent risk in allowing remote
> code to be downloaded in an ad-hoc manner like this, without any sort of
> vulnerability scanning, etc.
> As such, we should allow users to also package python packages in NiFi's
> native archiving format (NARs).
> The package structure should be as follows:
> {code:java}
> my-nar.nar
> +-- META-INF/
> +-- MANIFEST.MF
> +-- NAR-INF/
> +-- bundled-dependencies/
> +-- dependency1
> +-- dependency2
> +-- etc.
> +-- MyProcessor.py{code}
> Where {{MyProcessor.py}} could also be a python module / directory.
> In this way, we allow a Python Processor to be packaged up with its third
> party dependencies and dropped in the lib/ directory (or extensions)
> directory of a NiFi installation in the same way that a Java processor would
> be.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12959) Support loading Python processors from NARs
[
https://issues.apache.org/jira/browse/NIFI-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834435#comment-17834435
]
ASF subversion and git services commented on NIFI-12959:
Commit 2ad9db18db3d0ef0a1c0e3a9241e418299de78c3 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2ad9db18db ]
NIFI-12959: Support loading Python processors from NARs
This closes #8573
Signed-off-by: David Handermann
> Support loading Python processors from NARs
> ---
>
> Key: NIFI-12959
> URL: https://issues.apache.org/jira/browse/NIFI-12959
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently, third-party dependencies for Python Processors can be handled in
> two ways. Either they can be declared as dependencies in a Processor itself;
> or the Processor can be in a module where a {{requirements.txt}} dictates the
> requirements. These can be very helpful for developing Python based
> Processors.
> However, in production environments, it is not uncommon to see environments
> where {{pip}} is not installed. There is an inherent risk in allowing remote
> code to be downloaded in an ad-hoc manner like this, without any sort of
> vulnerability scanning, etc.
> As such, we should allow users to also package python packages in NiFi's
> native archiving format (NARs).
> The package structure should be as follows:
> {code:java}
> my-nar.nar
> +-- META-INF/
> +-- MANIFEST.MF
> +-- NAR-INF/
> +-- bundled-dependencies/
> +-- dependency1
> +-- dependency2
> +-- etc.
> +-- MyProcessor.py{code}
> Where {{MyProcessor.py}} could also be a python module / directory.
> In this way, we allow a Python Processor to be packaged up with its third
> party dependencies and dropped in the lib/ directory (or extensions)
> directory of a NiFi installation in the same way that a Java processor would
> be.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12959: Support loading python processors from NARs [nifi]
exceptionfactory closed pull request #8573: NIFI-12959: Support loading python processors from NARs URL: https://github.com/apache/nifi/pull/8573 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) ** Counter table has extra unnecessary can modify check (NIFI-12948) * Bulletin Board (NIFI-12560) * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) ** Clustering (NIFI-12807) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry (NIFI-12734) * Parameter Providers (NIFI-12622) ** Fetch parameters from provider, map to parameter context (dialog) - (NIFI-12665) * Cluster ** Node table (Disconnect/Connect/Load Balance/Etc) ** Status History - node specific values (NIFI-12848) * Flow Configuration History (NIFI-12754) ** ActionEntity.action should be optional (NIFI-12948) * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) ** Don't show users or groups in create/edit dialog is there are none (NIFI-12948) * Policies (NIFI-12548) ** Overridden policy Empty or Copy (NIFI-12679) ** Select Empty by default (NIFI-12948) * Help (NIFI-12795) * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) ** Clustering (NIFI-12807) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) ** Clustering * Change Component Version * Consider PG permissions in Toolbox (NIFI-12683) * Handle linking to components that are not on the canvas * PG Version (NIFI-12963 & NIFI-12995) ** Start (NIFI-12963) ** Commit (NIFI-12963) ** Force Commit (NIFI-12963) ** Show changes (NIFI-12995) ** Revert changes (NIFI-12995) ** Change Flow version (NIFI-12995) ** Stop (NIFI-12963) * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs ** Disable and Configure * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Stop and Configure ** Open Custom UI (NIFI-12958) ** Property History ** Unable to re-add any removed Property (NIFI-12743) ** Shift-Enter new line when editing Property (NIFI-12743) * Property Verification * More Details (Processor, Controller Service, Reporting Task) * Download Flow * Create RPG (NIFI-12758) * Configure RPG (NIFI-12774) * RPG Remote Ports (NIFI-12778) * RPG Go To (NIFI-12759) * RPG Refresh (NIFI-12761) * Color * Move to Front * Copy/Paste * Add/Update Info Icons in dialogs throughout the application * Set viewport earlier when loading a Process Group (NIFI-12737) * Canvas global menu item should navigate user back to where they were on the canvas (NIFI-12737) * Better theme support (NIFI-12655) * Set up development/production environments files * Run unit tests are part of standard build (NIFI-12941) * Update all API calls to consider disconnect node confirmation (NIFI-13001) * Update API calls to use uiOnly flag (NIFI-12950) * Use polling interval from API * Load FlowConfiguration in guard (NIFI-12948) * Routing error handling * General API response error handling ** Management CS (NIFI-12663) ** Canvas CS (NIFI-12684) ** Remainder of Settings (NIFI-12723) ** Counters (NIFI-12723) ** Bulletins (NIFI-12723) ** Flow Designer ** Parameter Contexts (NIFI-12937) ** Parameter ** Provenance (NIFI-12767) ** Queue Listing (NIFI-12742) ** Summary (NIFI-12742) ** Users (NIFI-12742) ** Policies ** Status History * Introduce header in new pages to unify with canvas and offer better navigation. (NIFI-12597) * Theme docs, view flow file, and custom ui's * Prompt user to save Parameter Context when Edit form is dirty * Upgrade to Angular 17 (NIFI-12790) * Start/Stop processors, process groups, ...
[jira] [Updated] (NIFI-13001) Disconnected Node Acknowledgement
[ https://issues.apache.org/jira/browse/NIFI-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13001: --- Status: Patch Available (was: Open) > Disconnected Node Acknowledgement > - > > Key: NIFI-13001 > URL: https://issues.apache.org/jira/browse/NIFI-13001 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When a node that is serving the UI for a client get's disconnected from the > cluster the user must acknowledge that they are disconnected prior to be able > to submit flow changes. Once they acknowledge the change, this acknowledge > must be sent in all mutable requests to the API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12889: Retry Kerberos login on auth failure in HDFS processors [nifi]
tpalfy commented on code in PR #8495:
URL: https://github.com/apache/nifi/pull/8495#discussion_r1554149017
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java:
##
@@ -352,95 +355,95 @@ protected void processBatchOfFiles(final List
files, final ProcessContext
for (final Path file : files) {
-ugi.doAs(new PrivilegedAction() {
-@Override
-public Object run() {
-FlowFile flowFile = session.create(parentFlowFile);
-try {
-final String originalFilename = file.getName();
-final Path outputDirPath = getNormalizedPath(context,
OUTPUT_DIRECTORY, parentFlowFile);
-final Path newFile = new Path(outputDirPath,
originalFilename);
-final boolean destinationExists = hdfs.exists(newFile);
-// If destination file already exists, resolve that
-// based on processor configuration
-if (destinationExists) {
-switch (processorConfig.getConflictResolution()) {
-case REPLACE_RESOLUTION:
-// Remove destination file (newFile) to
replace
-if (hdfs.delete(newFile, false)) {
-getLogger().info("deleted {} in order
to replace with the contents of {}",
-new Object[]{newFile,
flowFile});
-}
-break;
-case IGNORE_RESOLUTION:
-session.transfer(flowFile, REL_SUCCESS);
-getLogger().info(
-"transferring {} to success
because file with same name already exists",
-new Object[]{flowFile});
-return null;
-case FAIL_RESOLUTION:
-
session.transfer(session.penalize(flowFile), REL_FAILURE);
-getLogger().warn(
-"penalizing {} and routing to
failure because file with same name already exists",
-new Object[]{flowFile});
-return null;
-default:
-break;
-}
+ugi.doAs((PrivilegedAction) () -> {
Review Comment:
This way MoveHDFS is throwing a NullPointerException when the GSSException
occurs because it tries the next file after it did a rollback.
Here's a patch (against the current state of this PR) that would help:
```
Subject: [PATCH] Changes base on code review comments
---
Index:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===
diff --git
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java
---
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java
(revision 024860a2d8dccbaed8e98d280e1afd22dd0b7201)
+++
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java
(date 1712341636424)
@@ -47,9 +47,11 @@
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.util.StandardValidators;
import org.apache.nifi.util.StopWatch;
+import org.ietf.jgss.GSSException;
import java.io.FileNotFoundException;
import java.io.IOException;
+import java.io.UncheckedIOException;
import java.security.PrivilegedAction;
import java.security.PrivilegedExceptionAction;
import java.util.ArrayList;
@@ -57,6 +59,7 @@
import java.util.HashSet;
import java.util.List;
import java.util.Objects;
+import java.util.Optional;
import java.util.Set;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
@@ -325,16 +328,20 @@
queueLock.unlock();
}
-processBatchOfFiles(files, context, session, flowFile);
+try {
+processBatchOfFiles(files, context, session, flowFile);
+
Re: [PR] NIFI-12889: Retry Kerberos login on auth failure in HDFS processors [nifi]
tpalfy commented on code in PR #8495:
URL: https://github.com/apache/nifi/pull/8495#discussion_r1554121881
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/GetHDFS.java:
##
@@ -388,18 +388,18 @@ protected void processBatchOfFiles(final List
files, final ProcessContext
flowFile = session.putAttribute(flowFile,
CoreAttributes.FILENAME.key(), outputFilename);
if (!keepSourceFiles &&
!getUserGroupInformation().doAs((PrivilegedExceptionAction) () ->
hdfs.delete(file, false))) {
-getLogger().warn("Could not remove {} from HDFS. Not
ingesting this file ...",
-new Object[]{file});
+getLogger().warn("Could not remove {} from HDFS. Not
ingesting this file ...", file);
session.remove(flowFile);
continue;
}
session.getProvenanceReporter().receive(flowFile,
file.toString());
session.transfer(flowFile, REL_SUCCESS);
-getLogger().info("retrieved {} from HDFS {} in {} milliseconds
at a rate of {}",
-new Object[]{flowFile, file, millis, dataRate});
+getLogger().info("retrieved {} from HDFS {} in {} milliseconds
at a rate of {}", flowFile, file, millis, dataRate);
+} catch (final IOException e) {
Review Comment:
(GitHub doesn't allow me to suggest efficiently)
This way we don't handle non-GSS IOExceptions properly.
Instead of
```java
} catch (final IOException e) {
handleAuthErrors(e, session, context);
} catch (final Throwable t) {
getLogger().error("Error retrieving file {} from HDFS due to
{}", file, t);
session.rollback();
context.yield();
```
we should have
```java
} catch (final Throwable t) {
if (!handleAuthErrors(t, session, context)) {
getLogger().error("Error retrieving file {} from HDFS
due to {}", file, t);
session.rollback();
context.yield();
}
```
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java:
##
@@ -352,95 +355,95 @@ protected void processBatchOfFiles(final List
files, final ProcessContext
for (final Path file : files) {
-ugi.doAs(new PrivilegedAction() {
-@Override
-public Object run() {
-FlowFile flowFile = session.create(parentFlowFile);
-try {
-final String originalFilename = file.getName();
-final Path outputDirPath = getNormalizedPath(context,
OUTPUT_DIRECTORY, parentFlowFile);
-final Path newFile = new Path(outputDirPath,
originalFilename);
-final boolean destinationExists = hdfs.exists(newFile);
-// If destination file already exists, resolve that
-// based on processor configuration
-if (destinationExists) {
-switch (processorConfig.getConflictResolution()) {
-case REPLACE_RESOLUTION:
-// Remove destination file (newFile) to
replace
-if (hdfs.delete(newFile, false)) {
-getLogger().info("deleted {} in order
to replace with the contents of {}",
-new Object[]{newFile,
flowFile});
-}
-break;
-case IGNORE_RESOLUTION:
-session.transfer(flowFile, REL_SUCCESS);
-getLogger().info(
-"transferring {} to success
because file with same name already exists",
-new Object[]{flowFile});
-return null;
-case FAIL_RESOLUTION:
-
session.transfer(session.penalize(flowFile), REL_FAILURE);
-getLogger().warn(
-"penalizing {} and routing to
failure because file with same name already exists",
-new Object[]{flowFile});
-return null;
-default:
-break;
-}
+ugi.doAs((PrivilegedAction) () -> {
Review Comment:
This way
Re: [PR] NIFI-12918 Fix Stateless NullPointerException on versioned sub-process groups - main branch [nifi]
slambrose commented on PR #8536: URL: https://github.com/apache/nifi/pull/8536#issuecomment-2040356905 > Thanks for the latest updates @slambrose, this is looking closer to completion. > > Changing the `InMemoryFlowRegistry` to return `true` makes sense, as it is only used in Stateless operation, and it is the only Flow Registry Client available when running. It would be helpful to add a basic unit test class for `InMemoryFlowRegistry`, just to track that expected behavior. > > The other consideration is the base Registry URL. Although the current `getBaseRegistryUrl()` method works when a Registry server is accessible directly, if there is a reverse proxy in front that has an additional context path, rebuilding the URL with just the host and port presents a problem. For example: > > ``` > https://nifi-registry.local/context-path/nifi-registry-api > ``` > > The method would return the following: > > ``` > https://nifi-registry.local > ``` > > The Registry Client would then attempt to append `/nifi-registry-api` as the context path from the root, which would result in failures. > > In light of the fact that the `storageLocation` includes the full path, it seems like a regular expression match would address all scenarios. Something along the lines of the following pattern: > > ``` > ^(https?://.+?/nifi-registry-api).*$ > ``` > > With that pattern, the first capturing group can then be used at the registry URL. Additional Unit tests will be address in future work for InMemoryFlowRegistry class. Regex change has been added. Ready for re-review @exceptionfactory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12995) Remaining PG Versioning Actions
[ https://issues.apache.org/jira/browse/NIFI-12995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12995: --- Status: Patch Available (was: In Progress) > Remaining PG Versioning Actions > --- > > Key: NIFI-12995 > URL: https://issues.apache.org/jira/browse/NIFI-12995 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Change Flow Version > Show local changes > Revert local changes -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] [NIFI-12995]- Change Flow Version, Show Local Changes, Revert local changes [nifi]
rfellows opened a new pull request, #8607: URL: https://github.com/apache/nifi/pull/8607 [NIFI-12995](https://issues.apache.org/jira/browse/NIFI-12995) Also update the configuration around prettier so it will format html properly when it contains angular flow syntax. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-13001: Allowing the user to acknowledge changes to the cluster connection state [nifi]
mcgilman opened a new pull request, #8606: URL: https://github.com/apache/nifi/pull/8606 NIFI-13001: - Allowing the user to acknowledge changes to the cluster connection state. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2293 Support installing python dependencies defined inline [nifi-minifi-cpp]
szaszm commented on code in PR #1727: URL: https://github.com/apache/nifi-minifi-cpp/pull/1727#discussion_r1553992990 ## extensions/python/pythonprocessors/nifi_python_processors/utils/inline_dependency_installer.py: ## @@ -0,0 +1,49 @@ +import ast +import sys +import subprocess +import os + + +class Visitor(ast.NodeVisitor): Review Comment: ```suggestion # Extract the list of PIP dependency packages from the visited processor class AST node class Visitor(ast.NodeVisitor): ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2293 Support installing python dependencies defined inline [nifi-minifi-cpp]
szaszm commented on code in PR #1727:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1727#discussion_r1553987364
##
extensions/python/pythonprocessors/nifi_python_processors/utils/inline_dependency_installer.py:
##
@@ -0,0 +1,43 @@
+import ast
+import sys
+import subprocess
+import os
+
+
+class Visitor(ast.NodeVisitor):
+def __init__(self, class_name):
+self.dependencies = []
+self.class_name = class_name
+
+def visit_ClassDef(self, node):
+if node.name != self.class_name:
+return
+for child in node.body:
+if isinstance(child, ast.ClassDef) and child.name ==
'ProcessorDetails':
+for detail in child.body:
+if isinstance(detail, ast.Assign) and detail.targets[0].id
== 'dependencies':
+for elt in detail.value.elts:
+if isinstance(elt, ast.Constant):
+self.dependencies.append(elt.s)
+break
+break
+
+
+def extract_dependencies(file_path):
+class_name = file_path.split(os.sep)[-1].split('.')[0]
+with open(file_path, 'r') as file:
+code = file.read()
+
+tree = ast.parse(code)
+visitor = Visitor(class_name)
+visitor.visit(tree)
+return visitor.dependencies
+
+
+if __name__ == '__main__':
+if len(sys.argv) < 2:
+sys.exit(1)
+
+dependencies = extract_dependencies(sys.argv[1])
+if dependencies:
+subprocess.check_call([sys.executable, "-m", "pip", "install",
"--no-cache-dir"] + dependencies)
Review Comment:
It might be worth adding a code comment about this, possibly with reference
to NiFi's behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2277 Add virtualenv support for python processors [nifi-minifi-cpp]
szaszm commented on code in PR #1721: URL: https://github.com/apache/nifi-minifi-cpp/pull/1721#discussion_r1553978828 ## libminifi/src/Configuration.cpp: ## @@ -152,7 +152,11 @@ const std::unordered_map
[jira] [Updated] (NIFI-12958) Add support to open custom UIs
[
https://issues.apache.org/jira/browse/NIFI-12958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rob Fellows updated NIFI-12958:
---
Fix Version/s: 2.0.0-M3
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add support to open custom UIs
> --
>
> Key: NIFI-12958
> URL: https://issues.apache.org/jira/browse/NIFI-12958
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Allow the user to open a custom UI.
> 1) Open Custom UI from context menu. Not from Advanced button in Configure
> dialog.
> 2) Use route like the following...
> {noformat}
> /#/process-groups/43eb7385-018e-1000-2fb6-8dbbe88212cd/Processor/7bbab5aa-018e-1000-7de7-dc4052553b02/advanced{noformat}
> 3) Load component in question and using context path from component load UI
> in iframe using appropriate values for it's query params.
> 4) New page should use the header.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12958) Add support to open custom UIs
[
https://issues.apache.org/jira/browse/NIFI-12958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834368#comment-17834368
]
ASF subversion and git services commented on NIFI-12958:
Commit 2c706f5228152aa61f29d5135be293e62119c0cb in nifi's branch
refs/heads/main from Matt Gilman
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2c706f5228 ]
NIFI-12958: Adding support for custom UIs (#8601)
* NIFI-12958:
- Adding support for custom UIs.
- Running NiFi dev server at context path /nf.
- Fixing link used when clicking the logo in the header.
- Updating titles and icons used for editing components in Settings for better
consistency.
- Fixed JOLT advanced UI height.
* NIFI-12958:
- Fixing lint issue.
* NIFI-12958:
- Fixing test issue.
This closes #8601
> Add support to open custom UIs
> --
>
> Key: NIFI-12958
> URL: https://issues.apache.org/jira/browse/NIFI-12958
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Allow the user to open a custom UI.
> 1) Open Custom UI from context menu. Not from Advanced button in Configure
> dialog.
> 2) Use route like the following...
> {noformat}
> /#/process-groups/43eb7385-018e-1000-2fb6-8dbbe88212cd/Processor/7bbab5aa-018e-1000-7de7-dc4052553b02/advanced{noformat}
> 3) Load component in question and using context path from component load UI
> in iframe using appropriate values for it's query params.
> 4) New page should use the header.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12958: Adding support for custom UIs [nifi]
rfellows merged PR #8601: URL: https://github.com/apache/nifi/pull/8601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-13003) Service does not start if OIDC Metadata URL is unavailable
[
https://issues.apache.org/jira/browse/NIFI-13003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834365#comment-17834365
]
Igor Milavec edited comment on NIFI-13003 at 4/5/24 3:12 PM:
-
I understand and agree, but there should be some retry mechanism or (better) on
demand loading so that such (transient) errors do not cause NiFi service not to
start.
And debatably, logging on users is not the primary function of NiFi. :)
was (Author: JIRAUSER302515):
I understand and agree, but there should be some retry mechanism or (better) on
demand loading so that such (transient) errors do not cause NiFi service not to
start.
> Service does not start if OIDC Metadata URL is unavailable
> --
>
> Key: NIFI-13003
> URL: https://issues.apache.org/jira/browse/NIFI-13003
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration
>Affects Versions: 1.23.2
> Environment: Ubuntu 20.04 LTS, OpenJDK 17
>Reporter: Igor Milavec
>Priority: Major
>
> The NiFi service fails to start if it cannot retrieve the OpenID Connect
> metadata.
> The exception is:
>
> {code:java}
> Context initialization
> failedorg.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name
> 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration':
> Unsatisfied dependency expressed through method 'setFilterChains' parameter
> 0; nested exception is
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name 'securityFilterChain' defined in
> org.apache.nifi.web.security.configuration.WebSecurityConfiguration:
> Unsatisfied dependency expressed through method 'securityFilterChain'
> parameter 7; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating bean
> with name 'oAuth2LoginAuthenticationFilter' defined in
> org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
> instantiation via factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.springframework.security.oauth2.client.web.OAuth2LoginAuthenticationFilter]:
> Factory method 'oAuth2LoginAuthenticationFilter' threw exception; nested
> exception is org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'clientRegistrationRepository' defined in
> org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
> instantiation via factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.springframework.security.oauth2.client.registration.ClientRegistrationRepository]:
> Factory method 'clientRegistrationRepository' threw exception; nested
> exception is org.apache.nifi.web.security.oidc.OidcConfigurationException:
> OpenID Connect Metadata URL
> [https://login.microsoftonline.com/REDACTED/.well-known/openid-configuration]
> retrieval failedat
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.resolveMethodArguments(AutowiredAnnotationBeanPostProcessor.java:774)
>at
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:727)
>at
> org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
> at
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
>at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
>at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
>at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
>at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
>at
>
[jira] [Commented] (NIFI-13003) Service does not start if OIDC Metadata URL is unavailable
[
https://issues.apache.org/jira/browse/NIFI-13003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834365#comment-17834365
]
Igor Milavec commented on NIFI-13003:
-
I understand and agree, but there should be some retry mechanism or (better) on
demand loading so that such (transient) errors do not cause NiFi service not to
start.
> Service does not start if OIDC Metadata URL is unavailable
> --
>
> Key: NIFI-13003
> URL: https://issues.apache.org/jira/browse/NIFI-13003
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration
>Affects Versions: 1.23.2
> Environment: Ubuntu 20.04 LTS, OpenJDK 17
>Reporter: Igor Milavec
>Priority: Major
>
> The NiFi service fails to start if it cannot retrieve the OpenID Connect
> metadata.
> The exception is:
>
> {code:java}
> Context initialization
> failedorg.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name
> 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration':
> Unsatisfied dependency expressed through method 'setFilterChains' parameter
> 0; nested exception is
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name 'securityFilterChain' defined in
> org.apache.nifi.web.security.configuration.WebSecurityConfiguration:
> Unsatisfied dependency expressed through method 'securityFilterChain'
> parameter 7; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating bean
> with name 'oAuth2LoginAuthenticationFilter' defined in
> org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
> instantiation via factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.springframework.security.oauth2.client.web.OAuth2LoginAuthenticationFilter]:
> Factory method 'oAuth2LoginAuthenticationFilter' threw exception; nested
> exception is org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'clientRegistrationRepository' defined in
> org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
> instantiation via factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.springframework.security.oauth2.client.registration.ClientRegistrationRepository]:
> Factory method 'clientRegistrationRepository' threw exception; nested
> exception is org.apache.nifi.web.security.oidc.OidcConfigurationException:
> OpenID Connect Metadata URL
> [https://login.microsoftonline.com/REDACTED/.well-known/openid-configuration]
> retrieval failedat
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.resolveMethodArguments(AutowiredAnnotationBeanPostProcessor.java:774)
>at
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:727)
>at
> org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
> at
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
>at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
>at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
>at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
>at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
>at
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:921)
> at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
> at
>
[jira] [Commented] (NIFI-13003) Service does not start if OIDC Metadata URL is unavailable
[
https://issues.apache.org/jira/browse/NIFI-13003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834358#comment-17834358
]
David Handermann commented on NIFI-13003:
-
Thanks for providing the stack trace describing this issue [~imilavec].
This is expected behavior when NiFi is configured with OpenID Connect
authentication. Without access to the OIDC configuration, NiFi cannot attempt
to authenticate users, so a startup failure is expected.
> Service does not start if OIDC Metadata URL is unavailable
> --
>
> Key: NIFI-13003
> URL: https://issues.apache.org/jira/browse/NIFI-13003
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration
>Affects Versions: 1.23.2
> Environment: Ubuntu 20.04 LTS, OpenJDK 17
>Reporter: Igor Milavec
>Priority: Major
>
> The NiFi service fails to start if it cannot retrieve the OpenID Connect
> metadata.
> The exception is:
>
> {code:java}
> Context initialization
> failedorg.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name
> 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration':
> Unsatisfied dependency expressed through method 'setFilterChains' parameter
> 0; nested exception is
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name 'securityFilterChain' defined in
> org.apache.nifi.web.security.configuration.WebSecurityConfiguration:
> Unsatisfied dependency expressed through method 'securityFilterChain'
> parameter 7; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating bean
> with name 'oAuth2LoginAuthenticationFilter' defined in
> org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
> instantiation via factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.springframework.security.oauth2.client.web.OAuth2LoginAuthenticationFilter]:
> Factory method 'oAuth2LoginAuthenticationFilter' threw exception; nested
> exception is org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'clientRegistrationRepository' defined in
> org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
> instantiation via factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.springframework.security.oauth2.client.registration.ClientRegistrationRepository]:
> Factory method 'clientRegistrationRepository' threw exception; nested
> exception is org.apache.nifi.web.security.oidc.OidcConfigurationException:
> OpenID Connect Metadata URL
> [https://login.microsoftonline.com/REDACTED/.well-known/openid-configuration]
> retrieval failedat
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.resolveMethodArguments(AutowiredAnnotationBeanPostProcessor.java:774)
>at
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:727)
>at
> org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
> at
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
>at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
>at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
>at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
>at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
>at
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:921)
> at
>
[jira] [Updated] (NIFI-13004) Remove include-grpc from Developer Guide
[ https://issues.apache.org/jira/browse/NIFI-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Bean updated NIFI-13004: - Status: Patch Available (was: In Progress) > Remove include-grpc from Developer Guide > > > Key: NIFI-13004 > URL: https://issues.apache.org/jira/browse/NIFI-13004 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M2 >Reporter: Mark Bean >Assignee: Mark Bean >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The `include-grpc` profile was removed in NIFI-12069. However, a reference to > it still remains in the Developer Guide. Remove the reference. And check for > other references. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13004 remove include-grpc profile from developer and walkthrough… [nifi]
markobean opened a new pull request, #8605: URL: https://github.com/apache/nifi/pull/8605 … guides # Summary Simple update to the Developer and Walkthrough Guides to remove reference to `include-grpc` profile. This applies only to 2.x branch. Can the website be updated with this change immediately/prior to the next release? [NIFI-13004](https://issues.apache.org/jira/browse/NIFI-13004) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [X] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2323 ListenTCP custom delimiter [nifi-minifi-cpp]
szaszm commented on code in PR #1753: URL: https://github.com/apache/nifi-minifi-cpp/pull/1753#discussion_r1553773073 ## PROCESSORS.md: ## @@ -1561,6 +1561,8 @@ In the list below, the names of required properties appear in bold. Any other pr | **Max Size of Message Queue** | 1 || Maximum number of messages allowed to be buffered before processing them when the processor is triggered. If the buffer is full, the message is ignored. If set to zero the buffer is unlimited. | | SSL Context Service | || The Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection. | | Client Auth | NONE | NONEWANTREQUIRED | The client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided. | +| **Message Delimiter** | || The delimiter is used to divide the stream into flowfiles. | +| **Consume delimiter** | true | truefalse | If set to true then the delimiter won't be included in the resulting flowfiles. | Review Comment: A bit more info. ```suggestion | **Consume delimiter** | true | truefalse | If set to true then the delimiter won't be included at the end of the resulting flowfiles. | ``` ## PROCESSORS.md: ## @@ -1561,6 +1561,8 @@ In the list below, the names of required properties appear in bold. Any other pr | **Max Size of Message Queue** | 1 || Maximum number of messages allowed to be buffered before processing them when the processor is triggered. If the buffer is full, the message is ignored. If set to zero the buffer is unlimited. | | SSL Context Service | || The Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection. | | Client Auth | NONE | NONEWANTREQUIRED | The client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided. | +| **Message Delimiter** | || The delimiter is used to divide the stream into flowfiles. | Review Comment: The default value of '\n' is not visible, probably it's just not escaped. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-13004) Remove include-grpc from Developer Guide
[ https://issues.apache.org/jira/browse/NIFI-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Bean reassigned NIFI-13004: Assignee: Mark Bean > Remove include-grpc from Developer Guide > > > Key: NIFI-13004 > URL: https://issues.apache.org/jira/browse/NIFI-13004 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M2 >Reporter: Mark Bean >Assignee: Mark Bean >Priority: Major > > The `include-grpc` profile was removed in NIFI-12069. However, a reference to > it still remains in the Developer Guide. Remove the reference. And check for > other references. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13004) Remove include-grpc from Developer Guide
Mark Bean created NIFI-13004: Summary: Remove include-grpc from Developer Guide Key: NIFI-13004 URL: https://issues.apache.org/jira/browse/NIFI-13004 Project: Apache NiFi Issue Type: Bug Affects Versions: 2.0.0-M2 Reporter: Mark Bean The `include-grpc` profile was removed in NIFI-12069. However, a reference to it still remains in the Developer Guide. Remove the reference. And check for other references. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12999: Revert "Bump webpack-dev-middleware and karma-webpack (#8547)" [nifi]
sardell commented on PR #8603: URL: https://github.com/apache/nifi/pull/8603#issuecomment-2039894298 @exceptionfactory Thanks for sharing. I investigated a little more and I think I found the issue. There's a configuration property for npm called `engine-strict`. When set to true, version mismatches will cause a failure. I looked in my .npmrc file and saw mine was set to true. Rebuilding now after removing that property. If successful, I'll close this PR and issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-12960) ExcelReader - password protected XLSX
[ https://issues.apache.org/jira/browse/NIFI-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz reassigned NIFI-12960: --- Assignee: Daniel Stieglitz > ExcelReader - password protected XLSX > - > > Key: NIFI-12960 > URL: https://issues.apache.org/jira/browse/NIFI-12960 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Philipp Korniets >Assignee: Daniel Stieglitz >Priority: Trivial > > As we are trying to decomission ConvertExcelToCSVProcessor and replace it > with new ExcelReader it looks reasonable to implement functionality to read > password protected XLSX > Currently the only option is ExecuteScript with some code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12999: Revert "Bump webpack-dev-middleware and karma-webpack (#8547)" [nifi]
exceptionfactory commented on PR #8603:
URL: https://github.com/apache/nifi/pull/8603#issuecomment-2039800733
Thanks for providing additional details @sardell. As an additional point of
reference, I am able to build `nifi-registry-web-ui`, but NPM reports warnings,
not errors, for the version mismatch:
```
[INFO] --- frontend:1.14.2:npm (npm-install) @ nifi-registry-web-ui ---
[INFO] Running 'npm run ci' in
/Users/exceptionfactory/projects/nifi/nifi-registry/nifi-registry-core/nifi-registry-web-ui/target/frontend-working-directory
[INFO]
[INFO] > nifi-registry@0.0.5-SNAPSHOT ci
[INFO] > npm ci --ignore-scripts
[INFO]
[INFO] npm WARN EBADENGINE Unsupported engine {
[INFO] npm WARN EBADENGINE package: 'karma-webpack@5.0.1',
[INFO] npm WARN EBADENGINE required: { node: '>= 18' },
[INFO] npm WARN EBADENGINE current: { node: 'v16.13.2', npm: '8.1.2' }
[INFO] npm WARN EBADENGINE }
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12999: Revert "Bump webpack-dev-middleware and karma-webpack (#8547)" [nifi]
sardell commented on PR #8603: URL: https://github.com/apache/nifi/pull/8603#issuecomment-2039750043 I'm simply building the project from the root with `./mvnw clean install -T2C` (also tried without the parallel build). When I try to install npm dependencies by running `npm run ci` at `nifi/nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main` like the build does, I receive a dependency mismatch error. ``` nifi-registry-web-ui/src/main on NIFI-11520-with-backend [$] via v16.19.0 ❯ npm run ci > nifi-registry@0.0.5-SNAPSHOT ci > npm ci --ignore-scripts npm ERR! code ERESOLVE npm ERR! ERESOLVE could not resolve npm ERR! npm ERR! While resolving: @angular/http@4.4.7 npm ERR! Found: @angular/core@11.2.14 npm ERR! node_modules/@angular/core npm ERR! @angular/core@"11.2.14" from the root project npm ERR! peer @angular/core@"11.2.14" from @angular/animations@11.2.14 npm ERR! node_modules/@angular/animations npm ERR! @angular/animations@"11.2.14" from the root project npm ERR! peer @angular/animations@"^11.0.0 || ^12.0.0-0" from @angular/material@11.2.13 npm ERR! node_modules/@angular/material npm ERR! @angular/material@"11.2.13" from the root project npm ERR! 2 more (@covalent/core, @nifi-fds/core) npm ERR! 2 more (@angular/platform-browser, @nifi-fds/core) npm ERR! 10 more (@angular/cdk, @angular/common, @angular/flex-layout, ...) npm ERR! npm ERR! Could not resolve dependency: npm ERR! peer @angular/core@"4.4.7" from @angular/http@4.4.7 npm ERR! node_modules/@angular/http npm ERR! peer @angular/http@"^2.0.0||^4.0.0" from angular2-jwt@0.2.3 npm ERR! node_modules/angular2-jwt npm ERR! angular2-jwt@"0.2.3" from the root project npm ERR! npm ERR! Conflicting peer dependency: @angular/core@4.4.7 npm ERR! node_modules/@angular/core npm ERR! peer @angular/core@"4.4.7" from @angular/http@4.4.7 npm ERR! node_modules/@angular/http npm ERR! peer @angular/http@"^2.0.0||^4.0.0" from angular2-jwt@0.2.3 npm ERR! node_modules/angular2-jwt npm ERR! angular2-jwt@"0.2.3" from the root project npm ERR! npm ERR! Fix the upstream dependency conflict, or retry npm ERR! this command with --force, or --legacy-peer-deps npm ERR! to accept an incorrect (and potentially broken) dependency resolution. npm ERR! npm ERR! See /Users/sardell/.npm/eresolve-report.txt for a full report. npm ERR! A complete log of this run can be found in: npm ERR! /Users/sardell/.npm/_logs/2024-04-05T12_58_50_575Z-debug-0.log ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12958: Adding support for custom UIs [nifi]
rfellows commented on PR #8601: URL: https://github.com/apache/nifi/pull/8601#issuecomment-2039653644 will review... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13003) Service does not start if OIDC Metadata URL is unavailable
Igor Milavec created NIFI-13003:
---
Summary: Service does not start if OIDC Metadata URL is unavailable
Key: NIFI-13003
URL: https://issues.apache.org/jira/browse/NIFI-13003
Project: Apache NiFi
Issue Type: Bug
Components: Configuration
Affects Versions: 1.23.2
Environment: Ubuntu 20.04 LTS, OpenJDK 17
Reporter: Igor Milavec
The NiFi service fails to start if it cannot retrieve the OpenID Connect
metadata.
The exception is:
{code:java}
Context initialization
failedorg.springframework.beans.factory.UnsatisfiedDependencyException: Error
creating bean with name
'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration':
Unsatisfied dependency expressed through method 'setFilterChains' parameter 0;
nested exception is
org.springframework.beans.factory.UnsatisfiedDependencyException: Error
creating bean with name 'securityFilterChain' defined in
org.apache.nifi.web.security.configuration.WebSecurityConfiguration:
Unsatisfied dependency expressed through method 'securityFilterChain' parameter
7; nested exception is org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'oAuth2LoginAuthenticationFilter' defined in
org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
instantiation via factory method failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.springframework.security.oauth2.client.web.OAuth2LoginAuthenticationFilter]:
Factory method 'oAuth2LoginAuthenticationFilter' threw exception; nested
exception is org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'clientRegistrationRepository' defined in
org.apache.nifi.web.security.configuration.OidcSecurityConfiguration: Bean
instantiation via factory method failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.springframework.security.oauth2.client.registration.ClientRegistrationRepository]:
Factory method 'clientRegistrationRepository' threw exception; nested
exception is org.apache.nifi.web.security.oidc.OidcConfigurationException:
OpenID Connect Metadata URL
[https://login.microsoftonline.com/REDACTED/.well-known/openid-configuration]
retrieval failed at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.resolveMethodArguments(AutowiredAnnotationBeanPostProcessor.java:774)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:727)
at
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at
org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:921)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:399)
at
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:278)
at
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
at
[PR] MINIFICPP-2323 ListenTCP custom delimiter [nifi-minifi-cpp]
martinzink opened a new pull request, #1753: URL: https://github.com/apache/nifi-minifi-cpp/pull/1753 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-2319) Prevent RepoTests from creating file in the source tree
[ https://issues.apache.org/jira/browse/MINIFICPP-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink resolved MINIFICPP-2319. Resolution: Fixed https://github.com/apache/nifi-minifi-cpp/commit/34f640bb3c8c92365a77643a7aaa9ae8066392fc > Prevent RepoTests from creating file in the source tree > --- > > Key: MINIFICPP-2319 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2319 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Assignee: Ferenc Gerlits >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > A new test was added to RepoTests, which does not set MINIFI_HOME to a temp > directory, so it creates the bootstrap.conf file in the source tree. It > would be good to figure out how to prevent this from happening in the first > place, but for now, I'm just going to fix this one test. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2321) Splunk docker tests fail
[ https://issues.apache.org/jira/browse/MINIFICPP-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink resolved MINIFICPP-2321. Resolution: Fixed https://github.com/apache/nifi-minifi-cpp/commit/f533eb024f7587065d4fd50f9622b219630a6be1 > Splunk docker tests fail > > > Key: MINIFICPP-2321 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2321 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Splunk 9.0 image has been updated which broke our docker tests -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2321 Update Splunk tests for new docker image [nifi-minifi-cpp]
martinzink closed pull request #1750: MINIFICPP-2321 Update Splunk tests for new docker image URL: https://github.com/apache/nifi-minifi-cpp/pull/1750 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2319 Set the home directory on the Configure object in the unit test [nifi-minifi-cpp]
martinzink closed pull request #1747: MINIFICPP-2319 Set the home directory on the Configure object in the unit test URL: https://github.com/apache/nifi-minifi-cpp/pull/1747 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11858 Configurable Column Name Normalization in PutDatabaseRecord and UpdateDatabaseTable [nifi]
ravinarayansingh commented on PR #7544: URL: https://github.com/apache/nifi/pull/7544#issuecomment-2039271408 > @ravinarayansingh Thanks for your work and patience on this pull request. > > Unfortunately I may not have been clear in previous comments regarding the interface-based approached for the `ColumnNameNormalizer`. I did not intend for the `ColumnNameNormalizer` to be implemented by each Processor, but instead, the `ColumnNameNormalizer` should have separate implementations for each strategy. > > If it would be helpful, I could follow up with a commit that implements the approach I described. Hi @exceptionfactory I have made the required changes please have a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2314 - Send asset state hash in heartbeat, implement c2 asset sync [nifi-minifi-cpp]
martinzink commented on code in PR #1751:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1751#discussion_r1553149815
##
extensions/http-curl/tests/C2AssetSyncTest.cpp:
##
@@ -0,0 +1,257 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include
+#include
+#include
+#include
+
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "utils/IntegrationTestUtils.h"
+#include "utils/Environment.h"
+#include "utils/file/FileUtils.h"
+#include "utils/file/AssetManager.h"
+
+class FileProvider : public ServerAwareHandler {
Review Comment:
Just noting for visibility that this will cause some conflict with #1749
##
libminifi/src/core/state/nodes/AssetInformation.cpp:
##
@@ -0,0 +1,47 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "core/state/nodes/AssetInformation.h"
+#include "core/Resource.h"
+#include "core/logging/LoggerFactory.h"
+
+namespace org::apache::nifi::minifi::state::response {
+
+AssetInformation::AssetInformation()
+ : logger_(core::logging::LoggerFactory().getLogger()) {}
+
+void
AssetInformation::setAssetManager(std::shared_ptr
asset_manager) {
+ asset_manager_ = asset_manager;
+ if (!asset_manager_) {
+logger_->log_error("No asset manager is provided, asset information will
not be available");
+ }
+}
+
+std::vector AssetInformation::serialize() {
+ if (!asset_manager_) {
+return {};
+ }
+ SerializedResponseNode node;
+ node.name = "hash";
+ node.value = asset_manager_->hash();
+
+ return std::vector{node};
+}
+
+REGISTER_RESOURCE(AssetInformation, DescriptionOnly);
+
+} // namespace org::apache::nifi::minifi::state::response
Review Comment:
```suggestion
} // namespace org::apache::nifi::minifi::state::response
```
##
libminifi/src/core/state/nodes/AssetInformation.cpp:
##
@@ -0,0 +1,47 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "core/state/nodes/AssetInformation.h"
+#include "core/Resource.h"
+#include "core/logging/LoggerFactory.h"
+
+namespace org::apache::nifi::minifi::state::response {
+
+AssetInformation::AssetInformation()
+ : logger_(core::logging::LoggerFactory().getLogger()) {}
+
+void
AssetInformation::setAssetManager(std::shared_ptr
asset_manager) {
+ asset_manager_ = asset_manager;
Review Comment:
```suggestion
asset_manager_ = std::move(asset_manager);
```
##
extensions/http-curl/tests/C2AssetSyncTest.cpp:
##
@@ -0,0 +1,257 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for
[jira] [Resolved] (MINIFICPP-2322) MINIFI_INCLUDE_UCRT_DLLS and MINIFI_INCLUDE_VC_REDIST_DLLS shouldnt be mutually exclusive
[ https://issues.apache.org/jira/browse/MINIFICPP-2322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink resolved MINIFICPP-2322. Resolution: Fixed https://github.com/apache/nifi-minifi-cpp/commit/a2c22001b2835fe2386a3d202af6f484c75a2ecd > MINIFI_INCLUDE_UCRT_DLLS and MINIFI_INCLUDE_VC_REDIST_DLLS shouldnt be > mutually exclusive > - > > Key: MINIFICPP-2322 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2322 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[ https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834174#comment-17834174 ] Joe Witt commented on NIFI-12998: - Some TODOs: 1. remove the TODO entry by eliminating the nifi-nar-bom and just having the nifi-nar-bundle pom import the nifi-bom instead. 2. Do the entire build with all optional profiles to ensure everything is tidied up. 3. Once the full build is totally good/confirmed to have the right bits. THEN review ways to reduce version declarations without creating new needless coupling. 4. Evaluate whether certain poms could include/import the nifit-standard-services-api-bom to simplify version references or not. 5. review all the remaining things in the new build that were not in the old build. Some look like references we need to clean. API jars for example should not be in the nars/etc.. > nifi-nar-bundle has improper dependencies - the full source tree needs > dependency cleanup and management > > > Key: NIFI-12998 > URL: https://issues.apache.org/jira/browse/NIFI-12998 > Project: Apache NiFi > Issue Type: Task >Affects Versions: 2.0.0-M2 >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > found in nifi-nar-bundles pom > > com.maxmind.geoip2 > geoip2 > 4.2.0 > > This should not be here. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[ https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834173#comment-17834173 ] Joe Witt commented on NIFI-12998: - As for the nifi-python-framework-api it is properly ONLY in the resulting lib directory and not the other places. So again in the new build the output is more correct. > nifi-nar-bundle has improper dependencies - the full source tree needs > dependency cleanup and management > > > Key: NIFI-12998 > URL: https://issues.apache.org/jira/browse/NIFI-12998 > Project: Apache NiFi > Issue Type: Task >Affects Versions: 2.0.0-M2 >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > found in nifi-nar-bundles pom > > com.maxmind.geoip2 > geoip2 > 4.2.0 > > This should not be here. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[
https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834171#comment-17834171
]
Joe Witt commented on NIFI-12998:
-
In the new build the nifi-record lib is in the following places only. And the
main one that is important is nifi-standard-services-api-nar which all these
things extend from in a nar sense. So the new one looks a lot more correct.
need to review the ones that include it beyond the standard services api nar
and see if they can go away.
{noformat}
~/Downloads/new-nifi.txt:569:
./work/nar/extensions/nifi-couchbase-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
~/Downloads/new-nifi.txt:610:
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
~/Downloads/new-nifi.txt:627:
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
~/Downloads/new-nifi.txt:1260:
./work/nar/extensions/nifi-prometheus-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
~/Downloads/new-nifi.txt:1693:
./work/nar/extensions/nifi-standard-services-api-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
~/Downloads/new-nifi.txt:1787:
./work/nar/extensions/nifi-workday-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
{noformat}
> nifi-nar-bundle has improper dependencies - the full source tree needs
> dependency cleanup and management
>
>
> Key: NIFI-12998
> URL: https://issues.apache.org/jira/browse/NIFI-12998
> Project: Apache NiFi
> Issue Type: Task
>Affects Versions: 2.0.0-M2
>Reporter: Joe Witt
>Assignee: Joe Witt
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> found in nifi-nar-bundles pom
>
> com.maxmind.geoip2
> geoip2
> 4.2.0
>
> This should not be here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[
https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834167#comment-17834167
]
Joe Witt commented on NIFI-12998:
-
In Old (not new)
{noformat}
./work/nar/extensions/nifi-apicurio-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-aws-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-azure-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-box-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-confluent-platform-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dropbox-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-elasticsearch-client-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-elasticsearch-restapi-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-enrich-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-gcp-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-geohash-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-jms-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-jolt-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-kafka-2-6-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-lookup-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-mongodb-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-mongodb-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-mqtt-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-poi-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-py4j-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-record-serialization-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-record-sink-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-redis-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-registry-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-salesforce-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-scripting-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-site-to-site-reporting-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-slack-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-smb-client-api-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-smb-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-smb-smbj-client-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-standard-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-zendesk-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-zendesk-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-py4j-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-python-framework-api-2.0.0-SNAPSHOT.jar
./work/nar/framework/nifi-framework-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-python-framework-api-2.0.0-SNAPSHOT.jar
{noformat}
[jira] [Commented] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[
https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834166#comment-17834166
]
Joe Witt commented on NIFI-12998:
-
In New (not old)
{noformat}
./work/nar/extensions/nifi-amqp-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-distributed-cache-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-email-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-prometheus-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-ssl-context-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-websocket-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-websocket-services-api-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-websocket-services-jetty-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-amqp-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-asana-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-asana-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-compress-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-couchbase-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-distributed-cache-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-email-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-evtx-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-file-resource-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-framework-kubernetes-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-hazelcast-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-hl7-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-http-context-map-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-kerberos-credentials-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-kerberos-user-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-prometheus-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-proxy-configuration-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-server-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-single-user-iaa-providers-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-ssl-context-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-stateful-analysis-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-websocket-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-websocket-services-jetty-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-windows-event-log-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
[jira] [Commented] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[
https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834160#comment-17834160
]
Joe Witt commented on NIFI-12998:
-
to generate comparable output after creating a build start nifi. then in the
nifi bin root directory run
{noformat}
find . -type f | grep jar | sort -u
{noformat}
Do that on both old and new and you can compare the diffs.
> nifi-nar-bundle has improper dependencies - the full source tree needs
> dependency cleanup and management
>
>
> Key: NIFI-12998
> URL: https://issues.apache.org/jira/browse/NIFI-12998
> Project: Apache NiFi
> Issue Type: Task
>Affects Versions: 2.0.0-M2
>Reporter: Joe Witt
>Assignee: Joe Witt
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> found in nifi-nar-bundles pom
>
> com.maxmind.geoip2
> geoip2
> 4.2.0
>
> This should not be here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[
https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834159#comment-17834159
]
Joe Witt commented on NIFI-12998:
-
After latest commit added on the WIP pull request. The build compiles and
runs. But there is some remaining discrepancy between former build and new
build. The size is roughly the same but we need to resolve these differences.
Some look like gaps to solve and some look like improvements.
{noformat}
In New (not old)
./work/nar/extensions/nifi-amqp-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-amqp-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-ssl-context-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-amqp-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-asana-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-distributed-cache-client-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-asana-processors-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-asana-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-compress-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-couchbase-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-distributed-cache-client-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-couchbase-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-lookup-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-couchbase-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-serialization-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-couchbase-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-dbcp-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-json-schema-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-serialization-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-schema-registry-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-db-schema-registry-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-dbcp-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-kerberos-credentials-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-kerberos-user-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-serialization-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-record-sink-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-kerberos-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-dbcp-service-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-distributed-cache-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-distributed-cache-client-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-distributed-cache-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-distributed-cache-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-ssl-context-service-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-distributed-cache-services-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-utils-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-email-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-oauth2-provider-api-2.0.0-SNAPSHOT.jar
./work/nar/extensions/nifi-email-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-security-utils-api-2.0.0-SNAPSHOT.jar
[jira] [Updated] (NIFI-12998) nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management
[ https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12998: Summary: nifi-nar-bundle has improper dependencies - the full source tree needs dependency cleanup and management (was: maxmind geoip2 does not belong in nifi-nar-bundle pom) > nifi-nar-bundle has improper dependencies - the full source tree needs > dependency cleanup and management > > > Key: NIFI-12998 > URL: https://issues.apache.org/jira/browse/NIFI-12998 > Project: Apache NiFi > Issue Type: Task >Affects Versions: 2.0.0-M2 >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > found in nifi-nar-bundles pom > > com.maxmind.geoip2 > geoip2 > 4.2.0 > > This should not be here. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12923 Added append avro mode to PutHDFS [nifi]
balazsgerner commented on PR #8544:
URL: https://github.com/apache/nifi/pull/8544#issuecomment-2039019933
> In order for the append to work I had to set `Writing Strategy` to `Simple
write`, if I leave the default `Write and rename`, it actually deletes the
file. Is this intended? If not we should add another dependent property so if
AVRO is chosen, the Writing Strategy must be Simple Write.
To be honest, during the whole dev testing I left the 'Writing Strategy'
property on default value which is 'Write and rename', and it worked completely
fine for me. I double checked it now and it still does. As I see it, my ticket
does not affect the connection between writing strategy and conflict resolution
strategy so it should work just as before.
I also checked 'Writing strategy'='Simple write', and that also works for me
without any problems.
If you experience file deletion I assume your flow might need a tweak. I do
not know the processors you use exactly, but I had file deletion problems when
I left the 'GetHDFS' or the 'GetFile' processors 'Keep source file' on default
which is 'false'. In order to make my flow reexecutable I changed that value to
'true'.
For testing I used the following flow:
1. GetFile('Keep source file'='true') -> PutHDFS(append,AVRO,'Write and
rename'), but it also works with 'Simple write'
2. GetHDFS('Keep source file'='true') -> ConvertAvroToJSON (I just used this
step to validate the content of the avro file, not sure if the convert
processor exists upstream, I used a custom nifi instance for this)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
