[jira] [Updated] (NIFI-12962) Upgrade Spring Framework to 6.1.5
[ https://issues.apache.org/jira/browse/NIFI-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12962: Status: Patch Available (was: Open) > Upgrade Spring Framework to 6.1.5 > - > > Key: NIFI-12962 > URL: https://issues.apache.org/jira/browse/NIFI-12962 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 10m > Remaining Estimate: 0h > > Spring Framework dependencies should be upgraded to > [6.1.5|https://github.com/spring-projects/spring-framework/releases/tag/v6.1.5] > to align both NiFi and NiFi Registry on the same version. > Spring 6.1.0 introduced changes in AspectJ annotation expression parsing, > resulting in test failures for Auditor classes. Enabling parameters during > Java compilation as described in [explicit argument > names|https://docs.spring.io/spring-framework/reference/core/aop/ataspectj/advice.html#aop-ataspectj-advice-params-names-explicit] > documentation for Spring Framework resolves expression parsing and allows > the AspectJ annotations to work without changes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12962 Upgrade Spring Framework from 6.0.18 to 6.1.5 [nifi]
exceptionfactory opened a new pull request, #8575: URL: https://github.com/apache/nifi/pull/8575 # Summary [NIFI-12962](https://issues.apache.org/jira/browse/NIFI-12962) Upgrades Spring Framework dependencies from 6.0.18 to [6.1.5](https://github.com/spring-projects/spring-framework/releases/tag/v6.1.5) aligning the project root version with the current version used in NiFi Registry. This upgrade required enabling the [parameters](https://docs.oracle.com/en/java/javase/21/docs/specs/man/javac.html#option-parameters) option for the Java compiler in the `nifi-web-api` module. Enabling parameters supports runtime reflection and parameter name determination, which supports existing AspectJ Around expressions that require mapping expression arguments to method arguments. This approach follows the recommendation of the Spring Framework documentation for [explicit argument names](https://docs.spring.io/spring-framework/reference/core/aop/ataspectj/advice.html#aop-ataspectj-advice-params-names-explicit) and avoids the need to define and repeat the argument names in the annotations. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12962) Upgrade Spring Framework to 6.1.5
David Handermann created NIFI-12962: --- Summary: Upgrade Spring Framework to 6.1.5 Key: NIFI-12962 URL: https://issues.apache.org/jira/browse/NIFI-12962 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0-M3 Spring Framework dependencies should be upgraded to [6.1.5|https://github.com/spring-projects/spring-framework/releases/tag/v6.1.5] to align both NiFi and NiFi Registry on the same version. Spring 6.1.0 introduced changes in AspectJ annotation expression parsing, resulting in test failures for Auditor classes. Enabling parameters during Java compilation as described in [explicit argument names|https://docs.spring.io/spring-framework/reference/core/aop/ataspectj/advice.html#aop-ataspectj-advice-params-names-explicit] documentation for Spring Framework resolves expression parsing and allows the AspectJ annotations to work without changes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12961) Improve PutHDFS error handling
Lehel Boér created NIFI-12961: - Summary: Improve PutHDFS error handling Key: NIFI-12961 URL: https://issues.apache.org/jira/browse/NIFI-12961 Project: Apache NiFi Issue Type: Improvement Reporter: Lehel Boér Assignee: Lehel Boér Using return, break, throw, and so on from a finally block suppresses the propagation of any unhandled Throwable which was thrown in the try or catch block. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12776: Add UPLOAD provenance event to PutAzureBlobStorage [nifi]
Lehel44 closed pull request #8394: NIFI-12776: Add UPLOAD provenance event to PutAzureBlobStorage URL: https://github.com/apache/nifi/pull/8394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12959: Support loading python processors from NARs [nifi]
dan-s1 commented on code in PR #8573:
URL: https://github.com/apache/nifi/pull/8573#discussion_r1540151244
##
nifi-nar-bundles/nifi-py4j-bundle/nifi-py4j-bridge/src/test/java/org/apache/nifi/py4j/PythonProcessTest.java:
##
@@ -56,51 +56,46 @@ class PythonProcessTest {
@BeforeEach
public void setUp() {
-this.pythonProcess = new PythonProcess(this.pythonProcessConfig,
this.controllerServiceTypeLookup, virtualEnvHome, "Controller", "Controller");
+this.pythonProcess = new PythonProcess(this.pythonProcessConfig,
this.controllerServiceTypeLookup, virtualEnvHome, false, "Controller",
"Controller");
+}
+
+@Test
+void testUsesConfiguredValueWhenPackagedWithDependencies() throws
IOException {
+when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
+final PythonProcess process = new
PythonProcess(this.pythonProcessConfig, this.controllerServiceTypeLookup,
virtualEnvHome, true, "Controller", "Controller");
+assertEquals(PYTHON_CMD, process.resolvePythonCommand());
}
@Test
void testResolvePythonCommandWindows() throws IOException {
-final File scriptsDir = new File(virtualEnvHome, WINDOWS_SCRIPTS_DIR);
-assertTrue(scriptsDir.mkdir());
+final File scriptsDir = new File(virtualEnvHome, WINDOWS_SCRIPTS_DIR);
assertTrue(scriptsDir.mkdir());
-when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
-final String result = this.pythonProcess.resolvePythonCommand();
+when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
final String result = this.pythonProcess.resolvePythonCommand();
-final String expected = getExpectedBinaryPath(WINDOWS_SCRIPTS_DIR);
-assertEquals(expected, result);
+final String expected = getExpectedBinaryPath(WINDOWS_SCRIPTS_DIR);
assertEquals(expected, result);
}
@Test
void testResolvePythonCommandUnix() throws IOException {
-final File binDir = new File(virtualEnvHome, UNIX_BIN_DIR);
-assertTrue(binDir.mkdir());
+final File binDir = new File(virtualEnvHome, UNIX_BIN_DIR);
assertTrue(binDir.mkdir());
Review Comment:
```suggestion
final File binDir = new File(virtualEnvHome, UNIX_BIN_DIR);
assertTrue(binDir.mkdir());
```
##
nifi-nar-bundles/nifi-py4j-bundle/nifi-py4j-bridge/src/test/java/org/apache/nifi/py4j/PythonProcessTest.java:
##
@@ -56,51 +56,46 @@ class PythonProcessTest {
@BeforeEach
public void setUp() {
-this.pythonProcess = new PythonProcess(this.pythonProcessConfig,
this.controllerServiceTypeLookup, virtualEnvHome, "Controller", "Controller");
+this.pythonProcess = new PythonProcess(this.pythonProcessConfig,
this.controllerServiceTypeLookup, virtualEnvHome, false, "Controller",
"Controller");
+}
+
+@Test
+void testUsesConfiguredValueWhenPackagedWithDependencies() throws
IOException {
+when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
+final PythonProcess process = new
PythonProcess(this.pythonProcessConfig, this.controllerServiceTypeLookup,
virtualEnvHome, true, "Controller", "Controller");
+assertEquals(PYTHON_CMD, process.resolvePythonCommand());
}
@Test
void testResolvePythonCommandWindows() throws IOException {
-final File scriptsDir = new File(virtualEnvHome, WINDOWS_SCRIPTS_DIR);
-assertTrue(scriptsDir.mkdir());
+final File scriptsDir = new File(virtualEnvHome, WINDOWS_SCRIPTS_DIR);
assertTrue(scriptsDir.mkdir());
-when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
-final String result = this.pythonProcess.resolvePythonCommand();
+when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
final String result = this.pythonProcess.resolvePythonCommand();
-final String expected = getExpectedBinaryPath(WINDOWS_SCRIPTS_DIR);
-assertEquals(expected, result);
+final String expected = getExpectedBinaryPath(WINDOWS_SCRIPTS_DIR);
assertEquals(expected, result);
}
@Test
void testResolvePythonCommandUnix() throws IOException {
-final File binDir = new File(virtualEnvHome, UNIX_BIN_DIR);
-assertTrue(binDir.mkdir());
+final File binDir = new File(virtualEnvHome, UNIX_BIN_DIR);
assertTrue(binDir.mkdir());
-when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
-final String result = this.pythonProcess.resolvePythonCommand();
+when(pythonProcessConfig.getPythonCommand()).thenReturn(PYTHON_CMD);
final String result = this.pythonProcess.resolvePythonCommand();
-final String expected = getExpectedBinaryPath(UNIX_BIN_DIR);
-assertEquals(expected, result);
+final String expected = getExpectedBinaryPath(UNIX_BIN_DIR);
assertEquals(expected, result);
Review Comment:
[jira] [Updated] (NIFI-12940) Migrate to Nx
[ https://issues.apache.org/jira/browse/NIFI-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12940: --- Status: Patch Available (was: In Progress) > Migrate to Nx > - > > Key: NIFI-12940 > URL: https://issues.apache.org/jira/browse/NIFI-12940 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12923 Added append avro mode to PutHDFS [nifi]
Lehel44 commented on code in PR #8544:
URL: https://github.com/apache/nifi/pull/8544#discussion_r1540085973
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java:
##
@@ -160,6 +170,14 @@ public class PutHDFS extends AbstractHadoopProcessor {
.allowableValues(WRITE_AND_RENAME_AV, SIMPLE_WRITE_AV)
.build();
+public static final PropertyDescriptor APPEND_MODE = new
PropertyDescriptor.Builder()
+.name("Append mode")
Review Comment:
```suggestion
.name("Append Mode")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12889: Retry Kerberos login on auth failure in HDFS processors [nifi]
Lehel44 commented on code in PR #8495:
URL: https://github.com/apache/nifi/pull/8495#discussion_r1538284672
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java:
##
@@ -465,10 +465,16 @@ public void process(InputStream in) throws IOException {
} catch (final IOException e) {
Review Comment:
Since we throw the GSSException wrapped in a ProcessException, I think we
have to catch the ProcessException, then check if it's a GSSException. If so
rollback, othwerwise throw it away.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12950: Updating front end to use the uiOnly flag where possible [nifi]
rfellows commented on code in PR #8565:
URL: https://github.com/apache/nifi/pull/8565#discussion_r1540073157
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -1069,6 +1069,16 @@ export class FlowEffects {
this.actions$.pipe(
ofType(FlowActions.openEditProcessorDialog),
map((action) => action.request),
+switchMap((request) =>
+
from(this.flowService.getProcessor(request.entity.id)).pipe(
+map((entity) => {
+return {
+...request,
+entity
+};
+})
+)
Review Comment:
I'm fine with not addressing it here as long it is accounted for in the near
future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12950: Updating front end to use the uiOnly flag where possible [nifi]
rfellows commented on code in PR #8565:
URL: https://github.com/apache/nifi/pull/8565#discussion_r1540073157
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -1069,6 +1069,16 @@ export class FlowEffects {
this.actions$.pipe(
ofType(FlowActions.openEditProcessorDialog),
map((action) => action.request),
+switchMap((request) =>
+
from(this.flowService.getProcessor(request.entity.id)).pipe(
+map((entity) => {
+return {
+...request,
+entity
+};
+})
+)
Review Comment:
I'm fine with not addressing it here as long it is accounted for.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12939) Retry Kerberos login on authentication failure in Iceberg processors
[ https://issues.apache.org/jira/browse/NIFI-12939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-12939: Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Retry Kerberos login on authentication failure in Iceberg processors > > > Key: NIFI-12939 > URL: https://issues.apache.org/jira/browse/NIFI-12939 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 40m > Remaining Estimate: 0h > > When multiple processors trying to renew the same expired Kerberos ticket, it > can cause authentication error that will route the current FlowFile into the > processor's failure relation. Since the issue is not permanent the processor > should retry the authentication. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12939) Retry Kerberos login on authentication failure in Iceberg processors
[ https://issues.apache.org/jira/browse/NIFI-12939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831064#comment-17831064 ] ASF subversion and git services commented on NIFI-12939: Commit 2a5b61136b251fbbcda40ccc39de0be29892e497 in nifi's branch refs/heads/main from Mark Bathori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2a5b61136b ] NIFI-12939: Retry Kerberos login on authentication failure in Iceberg processors Update rollback and context yield calls Signed-off-by: Matt Burgess This closes #8553 > Retry Kerberos login on authentication failure in Iceberg processors > > > Key: NIFI-12939 > URL: https://issues.apache.org/jira/browse/NIFI-12939 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > When multiple processors trying to renew the same expired Kerberos ticket, it > can cause authentication error that will route the current FlowFile into the > processor's failure relation. Since the issue is not permanent the processor > should retry the authentication. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12939) Retry Kerberos login on authentication failure in Iceberg processors
[ https://issues.apache.org/jira/browse/NIFI-12939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831063#comment-17831063 ] ASF subversion and git services commented on NIFI-12939: Commit 04b8587dfdf4c785b87035b64ff4ec1eb983645e in nifi's branch refs/heads/support/nifi-1.x from Mark Bathori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=04b8587dfd ] NIFI-12939: Retry Kerberos login on authentication failure in Iceberg processors Update rollback and context yield calls Signed-off-by: Matt Burgess > Retry Kerberos login on authentication failure in Iceberg processors > > > Key: NIFI-12939 > URL: https://issues.apache.org/jira/browse/NIFI-12939 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > When multiple processors trying to renew the same expired Kerberos ticket, it > can cause authentication error that will route the current FlowFile into the > processor's failure relation. Since the issue is not permanent the processor > should retry the authentication. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12939: Retry Kerberos login on authentication failure in Iceberg processors [nifi]
mattyb149 closed pull request #8553: NIFI-12939: Retry Kerberos login on authentication failure in Iceberg processors URL: https://github.com/apache/nifi/pull/8553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12255 refactor PutElasticsearchRecord and PutElasticsearchJson relationships to be more consistent with other processors [nifi]
exceptionfactory commented on code in PR #7940:
URL: https://github.com/apache/nifi/pull/7940#discussion_r1540029633
##
nifi-mock/src/main/java/org/apache/nifi/util/StandardProcessorTestRunner.java:
##
@@ -229,33 +226,32 @@ public void run(final int iterations, final boolean
stopOnFinish, final boolean
executorService.awaitTermination(runWait,
TimeUnit.MILLISECONDS);
} catch (final InterruptedException e1) {
}
+}
-int finishedCount = 0;
-boolean unscheduledRun = false;
-for (final Future future : futures) {
-try {
-final Throwable thrown = future.get(); // wait for the
result
-if (thrown != null) {
-throw new AssertionError(thrown);
-}
+int finishedCount = 0;
+boolean unscheduledRun = false;
+for (final Future future : futures) {
+try {
+final Throwable thrown = future.get(); // wait for the result
+if (thrown != null) {
+throw new AssertionError(thrown);
+}
-if (++finishedCount == 1 && stopOnFinish) {
-unscheduledRun = true;
-unSchedule();
-}
-} catch (final Exception e) {
+if (++finishedCount == 1 && stopOnFinish) {
+unscheduledRun = true;
+unSchedule();
}
+} catch (final Exception e) {
+e.printStackTrace();
Review Comment:
Thanks, I think it is best to avoid the `printStackTrace()` for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12959) Support loading Python processors from NARs
[
https://issues.apache.org/jira/browse/NIFI-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-12959:
--
Status: Patch Available (was: Open)
> Support loading Python processors from NARs
> ---
>
> Key: NIFI-12959
> URL: https://issues.apache.org/jira/browse/NIFI-12959
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, third-party dependencies for Python Processors can be handled in
> two ways. Either they can be declared as dependencies in a Processor itself;
> or the Processor can be in a module where a {{requirements.txt}} dictates the
> requirements. These can be very helpful for developing Python based
> Processors.
> However, in production environments, it is not uncommon to see environments
> where {{pip}} is not installed. There is an inherent risk in allowing remote
> code to be downloaded in an ad-hoc manner like this, without any sort of
> vulnerability scanning, etc.
> As such, we should allow users to also package python packages in NiFi's
> native archiving format (NARs).
> The package structure should be as follows:
> {code:java}
> my-nar.nar
> +-- META-INF/
> +-- MANIFEST.MF
> +-- NAR-INF/
> +-- bundled-dependencies/
> +-- dependency1
> +-- dependency2
> +-- etc.
> +-- MyProcessor.py{code}
> Where {{MyProcessor.py}} could also be a python module / directory.
> In this way, we allow a Python Processor to be packaged up with its third
> party dependencies and dropped in the lib/ directory (or extensions)
> directory of a NiFi installation in the same way that a Java processor would
> be.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12959: Support loading python processors from NARs [nifi]
markap14 opened a new pull request, #8573: URL: https://github.com/apache/nifi/pull/8573 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12940) Migrate to Nx
[ https://issues.apache.org/jira/browse/NIFI-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12940: --- Summary: Migrate to Nx (was: Migrate from Karma to Jest) > Migrate to Nx > - > > Key: NIFI-12940 > URL: https://issues.apache.org/jira/browse/NIFI-12940 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) ** Counter table has extra unnecessary can modfiy check * Bulletin Board (NIFI-12560) ** When the bulletin board polls it is missing the offset "after" query parameter * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) ** Clustering (NIFI-12807) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry (NIFI-12734) * Parameter Providers (NIFI-12622) ** Fetch parameters from provider, map to parameter context (dialog) - (NIFI-12665) * Cluster ** Status History - node specific values (NIFI-12848) * Flow Configuration History (NIFI-12754) ** ActionEntity.action should be optional * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) ** Don't show users or groups in create/edit dialog is there are none * Policies (NIFI-12548) ** Overridden policy Empty or Copy (NIFI-12679) ** Select Empty by default * Help (NIFI-12795) * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) ** Clustering (NIFI-12807) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) ** Clustering * Change Component Version * Consider PG permissions in Toolbox (NIFI-12683) * Handle linking to components that are not on the canvas * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs ** Disable and Configure * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Stop and Configure ** Open Custom UI (NIFI-12958) ** Property History ** Unable to re-add any removed Property (NIFI-12743) ** Shift-Enter new line when editing Property (NIFI-12743) * Property Verification * More Details (Processor, Controller Service, Reporting Task) * Download Flow * Create RPG (NIFI-12758) * Configure RPG (NIFI-12774) * RPG Remote Ports (NIFI-12778) * RPG Go To (NIFI-12759) * RPG Refresh (NIFI-12761) * Color * Move to Front * Copy/Paste * Add/Update Info Icons in dialogs throughout the application * Set viewport earlier when loading a Process Group (NIFI-12737) * Canvas global menu item should navigate user back to where they were on the canvas (NIFI-12737) * Better theme support (NIFI-12655) * Set up development/production environments files * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmation * Update API calls to use uiOnly flag * Use polling interval from API * Load FlowConfiguration in guard * Routing error handling * General API response error handling ** Management CS (NIFI-12663) ** Canvas CS (NIFI-12684) ** Remainder of Settings (NIFI-12723) ** Counters (NIFI-12723) ** Bulletins (NIFI-12723) ** Flow Designer ** Parameter Contexts (NIFI-12937) ** Provenance (NIFI-12767) ** Queue Listing (NIFI-12742) ** Summary (NIFI-12742) ** Users (NIFI-12742) ** Policies ** Status History * Introduce header in new pages to unify with canvas and offer better navigation. (NIFI-12597) * Theme docs, view flow file, and custom ui's * Prompt user to save Parameter Context when Edit form is dirty * Upgrade to Angular 17 (NIFI-12790) * Start/Stop processors, process groups, ... (NIFI-12568) * Dialog vertical resizing on smaller screens do not allow users to access all fields (NIFI-12603) * Flow Analysis report menu * Migrate NiFi UI to nx (NIFI-12940) * Migrate from Karma tests to
Re: [PR] NIFI-12950: Updating front end to use the uiOnly flag where possible [nifi]
mcgilman commented on code in PR #8565:
URL: https://github.com/apache/nifi/pull/8565#discussion_r1539994109
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -1069,6 +1069,16 @@ export class FlowEffects {
this.actions$.pipe(
ofType(FlowActions.openEditProcessorDialog),
map((action) => action.request),
+switchMap((request) =>
+
from(this.flowService.getProcessor(request.entity.id)).pipe(
+map((entity) => {
+return {
+...request,
+entity
+};
+})
+)
Review Comment:
We do need error handling here, however we haven't considered it yet in
`FlowEffects`. There is a line item for it in NIFI-12400. That said, if you'd
prefer I cover this case here now I will.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12255 refactor PutElasticsearchRecord and PutElasticsearchJson relationships to be more consistent with other processors [nifi]
ChrisSamo632 commented on code in PR #7940:
URL: https://github.com/apache/nifi/pull/7940#discussion_r1539989744
##
nifi-mock/src/main/java/org/apache/nifi/util/StandardProcessorTestRunner.java:
##
@@ -229,33 +226,32 @@ public void run(final int iterations, final boolean
stopOnFinish, final boolean
executorService.awaitTermination(runWait,
TimeUnit.MILLISECONDS);
} catch (final InterruptedException e1) {
}
+}
-int finishedCount = 0;
-boolean unscheduledRun = false;
-for (final Future future : futures) {
-try {
-final Throwable thrown = future.get(); // wait for the
result
-if (thrown != null) {
-throw new AssertionError(thrown);
-}
+int finishedCount = 0;
+boolean unscheduledRun = false;
+for (final Future future : futures) {
+try {
+final Throwable thrown = future.get(); // wait for the result
+if (thrown != null) {
+throw new AssertionError(thrown);
+}
-if (++finishedCount == 1 && stopOnFinish) {
-unscheduledRun = true;
-unSchedule();
-}
-} catch (final Exception e) {
+if (++finishedCount == 1 && stopOnFinish) {
+unscheduledRun = true;
+unSchedule();
}
+} catch (final Exception e) {
+e.printStackTrace();
Review Comment:
The `Exception` capture is nothing new, just previously it was being
silently ignored, so I thought I'd at least indicate that the `catch` had been
triggered, without potentially causing problems for lots of tests across the
existing estate
Happy to revert this and silently swallow the `InterruptedException` &
`ExecutionException`s, although I'll change it from catching all `Exception`s
at the least
This approach matches other similar handling elsewhere within the
`TestRunner`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12950: Updating front end to use the uiOnly flag where possible [nifi]
rfellows commented on code in PR #8565:
URL: https://github.com/apache/nifi/pull/8565#discussion_r1539977978
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -1069,6 +1069,16 @@ export class FlowEffects {
this.actions$.pipe(
ofType(FlowActions.openEditProcessorDialog),
map((action) => action.request),
+switchMap((request) =>
+
from(this.flowService.getProcessor(request.entity.id)).pipe(
+map((entity) => {
+return {
+...request,
+entity
+};
+})
+)
Review Comment:
Do we need to consider the possibility of this endpoint failing?
When I force an error in the service and try to edit a processor config,
nothing happens. Should we pop up a snackbar in this rare case? Or is ignoring
it sufficient?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12948) UI - Address minor bugs discovered during Angular migration
[ https://issues.apache.org/jira/browse/NIFI-12948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831052#comment-17831052 ] ASF subversion and git services commented on NIFI-12948: Commit 0c01dfe58501fa86a43621cad60cef03a364a40b in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0c01dfe585 ] NIFI-12948: (#8564) - Action is optional. - Removing duplicate check in counter table. - Ensure flow configuration is loaded in route guard. - Only show user/group list when there are users or groups. - Default override policy to copy. > UI - Address minor bugs discovered during Angular migration > --- > > Key: NIFI-12948 > URL: https://issues.apache.org/jira/browse/NIFI-12948 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When migrating to Angular 17 and reviewing the control flow changes a handful > a bugs were discovered. This Jira is to address each of them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12948: Various bugs discovered when migrating to Angular 17 [nifi]
scottyaslan merged PR #8564: URL: https://github.com/apache/nifi/pull/8564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12950: Updating front end to use the uiOnly flag where possible [nifi]
rfellows commented on PR #8565: URL: https://github.com/apache/nifi/pull/8565#issuecomment-2021277121 reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12937) Error Handling in Parameter Context Page
[ https://issues.apache.org/jira/browse/NIFI-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831049#comment-17831049 ] ASF subversion and git services commented on NIFI-12937: Commit 339a06a8c6337d290361acb4c57be348930ff9c5 in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=339a06a8c6 ] NIFI-12937: Adding error handling to the Parameter Context Listing page (#8555) * NIFI-12937: - Adding error handling to the Parameter Context Listing page. * NIFI-12937: - Addressing review feedback. This closes #8555 > Error Handling in Parameter Context Page > > > Key: NIFI-12937 > URL: https://issues.apache.org/jira/browse/NIFI-12937 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12939: Retry Kerberos login on authentication failure in Iceberg processors [nifi]
mattyb149 commented on PR #8553: URL: https://github.com/apache/nifi/pull/8553#issuecomment-2021266007 +1 LGTM, thanks for the improvement! Merging to main and support/nifi-1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12937: Adding error handling to the Parameter Context Listing page [nifi]
rfellows merged PR #8555: URL: https://github.com/apache/nifi/pull/8555 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12937) Error Handling in Parameter Context Page
[ https://issues.apache.org/jira/browse/NIFI-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12937: --- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Error Handling in Parameter Context Page > > > Key: NIFI-12937 > URL: https://issues.apache.org/jira/browse/NIFI-12937 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12941) Migrate from Karma to Jest
[ https://issues.apache.org/jira/browse/NIFI-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831045#comment-17831045 ] ASF subversion and git services commented on NIFI-12941: Commit 039fd66911b374db3a05629e94da648285386503 in nifi's branch refs/heads/main from Rob Fellows [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=039fd66911 ] [NIFI-12941] - Migrate to Jest from Karma/Jasmine. (#8566) * [NIFI-12941] - Migrate to Jest from Karma/Jasmine. * Updated license header * Review Feedback * remove jest transforms that were not needed This closes #8566 > Migrate from Karma to Jest > -- > > Key: NIFI-12941 > URL: https://issues.apache.org/jira/browse/NIFI-12941 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12941) Migrate from Karma to Jest
[ https://issues.apache.org/jira/browse/NIFI-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12941: --- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Migrate from Karma to Jest > -- > > Key: NIFI-12941 > URL: https://issues.apache.org/jira/browse/NIFI-12941 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12941] - Migrate to Jest from Karma/Jasmine. [nifi]
mcgilman merged PR #8566: URL: https://github.com/apache/nifi/pull/8566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11671) JoinEnrichment SQL strategy doesn't allow attributes in join statement
[
https://issues.apache.org/jira/browse/NIFI-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831042#comment-17831042
]
Philipp Korniets commented on NIFI-11671:
-
[~markap14] , [~pvillard] - any update please?
> JoinEnrichment SQL strategy doesn't allow attributes in join statement
> --
>
> Key: NIFI-11671
> URL: https://issues.apache.org/jira/browse/NIFI-11671
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.18.0, 1.20.0, 1.23.0
>Reporter: Philipp Korniets
>Assignee: Mark Payne
>Priority: Minor
> Attachments: screenshot-1.png, screenshot-2.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We use ForkEnrichement - JoinEnrichment pattern and want to include filtering
> in join SQL. Filter value is coming from FlowFile attribute
> {code:sql}
> ${test} = 'NewValue'
> SELECT original.*, enrichment.*,'${test}'
> FROM original
> LEFT OUTER JOIN enrichment
> ON original.Underlying = enrichment.Underlying
> WHERE enrichment.MyField = '${test}'
> {code}
> However this doesnt work because JoinEnrichment doesnt use
> evaluateAttributeExpressions
> Additionally in version 1.18,1.23 - doesnt allow whole query to be passed as
> attribute.
> !screenshot-1.png|width=692,height=431!
>
> {code:java}
> 2023-06-28 11:07:16,611 ERROR [Timer-Driven Process Thread-7]
> o.a.n.processors.standard.JoinEnrichment
> JoinEnrichment[id=dbe156ac-0187-1000-4477-0183899e0432] Failed to join
> 'original' FlowFile
> StandardFlowFileRecord[uuid=2ab9f6ad-73a5-4763-b25e-fd26c44835e1,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1687948831976-629,
> container=default, section=629], offset=8334082,
> length=600557],offset=0,name=lmr_SY08C41-1_S_514682_20230627.csv,size=600557]
> and 'enrichment' FlowFile
> StandardFlowFileRecord[uuid=e4bb7769-fdce-4dfe-af18-443676103035,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1687949723375-631,
> container=default, section=631], offset=5362822,
> length=1999502],offset=0,name=lmr_SY08C41-1_S_514682_20230627.csv,size=1999502];
> routing to failure
> java.sql.SQLException: Error while preparing statement [${instrumentJoinSQL}]
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:224)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:203)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:99)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareStatement(AvaticaConnection.java:178)
> at
> org.apache.nifi.processors.standard.enrichment.SqlJoinCache.createCalciteParameters(SqlJoinCache.java:91)
> at
> org.apache.nifi.processors.standard.enrichment.SqlJoinCache.getCalciteParameters(SqlJoinCache.java:65)
> at
> org.apache.nifi.processors.standard.enrichment.SqlJoinStrategy.join(SqlJoinStrategy.java:49)
> at
> org.apache.nifi.processors.standard.JoinEnrichment.processBin(JoinEnrichment.java:387)
> at
> org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:233)
> at
> org.apache.nifi.processors.standard.JoinEnrichment.processBins(JoinEnrichment.java:503)
> at
> org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:193)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1354)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.RuntimeException: parse failed: Encountered "$" at line
> 1, column 1.
> Was expecting one of:
> "ABS" ...
> {code}
> As I understand issue is in following line of code
>
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831039#comment-17831039 ] Philipp Korniets commented on NIFI-11167: - [~dstiegli1] - https://issues.apache.org/jira/browse/NIFI-12960 > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0-M1, 1.23.0 > > Attachments: CSVRecordSetWriter_configuration.png, > ExcelReaderConfiguration.png, QueryRecord_configuration.png, Test > ExcelReader.xlsx, Test Sheet-formula.xlsx, image-2023-11-28-18-22-07-446.png, > image-2023-11-29-15-51-08-386.png, resulting.csv, screenshot-1.png > > Time Spent: 10h 10m > Remaining Estimate: 0h > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12960) ExcelReader - password protected XLSX
Philipp Korniets created NIFI-12960: --- Summary: ExcelReader - password protected XLSX Key: NIFI-12960 URL: https://issues.apache.org/jira/browse/NIFI-12960 Project: Apache NiFi Issue Type: New Feature Components: Core Framework Affects Versions: 1.25.0 Reporter: Philipp Korniets As we are trying to decomission ConvertExcelToCSVProcessor and replace it with new ExcelReader it looks reasonable to implement functionality to read password protected XLSX Currently the only option is ExecuteScript with some code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12918 Fix Stateless NullPointerException on versioned sub-process groups - 1.x support branch [nifi]
slambrose commented on PR #8572: URL: https://github.com/apache/nifi/pull/8572#issuecomment-2021186177 Okay, I'm trying one more solution. What I've found is that the main branch has new code where a "registryId" is implemented, but it is always null right now because the registry-api does not currently supply a "registryId" property value. The 1.x support branch does not have this new class. I submitted the fix in the PR to the main branch, but for 1.x, I have something working in the temporary until the changes from 2.0 are pushed down to 1.x (if they are going to do that). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12918 Fix Stateless NullPointerException on versioned sub-process groups - 1.x support branch [nifi]
slambrose opened a new pull request, #8572: URL: https://github.com/apache/nifi/pull/8572 # Summary [NIFI-12918](https://issues.apache.org/jira/browse/NIFI-12918) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [X] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [X] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [X] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-12922] semantic dialog size configurations [nifi]
scottyaslan commented on code in PR #8535:
URL: https://github.com/apache/nifi/pull/8535#discussion_r1539869502
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/ui/common/yes-no-dialog/yes-no-dialog.component.html:
##
@@ -17,7 +17,7 @@
{{ request.title }}
-{{ request.message
}}
Review Comment:
Ok, here is the current approach:
- YesNoDialog's are now all SMALL_DIALOG's now which get the `max-width:
24rem`.
- OkDialogs can be any size dialog but should also have `maxWidth: '24rem'`
override when opening the dialog
- Cancel dialog messages are controlled by the frontend and are small. No
dialog size or max width are set on these
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12918 Fix Stateless NullPointerException on versioned sub-process groups - main branch [nifi]
slambrose commented on PR #8536: URL: https://github.com/apache/nifi/pull/8536#issuecomment-2021173210 Okay, I'm trying one more solution. What I've found is that the main branch has new code where a "registryId" is implemented, but it is always null right now because the registry-api does not currently supply a "registryId" property value. The integration tests hard-code the creation of a registry UUID to simulate testing the bug that currently exists in 1.x and 2.0 (main) branches. This will put in a UUID placeholder and allow the integration tests to pass, while also fixing the current broken stateless flows in 1.x and 2.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12959) Support loading Python processors from NARs
Mark Payne created NIFI-12959:
-
Summary: Support loading Python processors from NARs
Key: NIFI-12959
URL: https://issues.apache.org/jira/browse/NIFI-12959
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 2.0.0-M3
Currently, third-party dependencies for Python Processors can be handled in two
ways. Either they can be declared as dependencies in a Processor itself; or the
Processor can be in a module where a {{requirements.txt}} dictates the
requirements. These can be very helpful for developing Python based Processors.
However, in production environments, it is not uncommon to see environments
where {{pip}} is not installed. There is an inherent risk in allowing remote
code to be downloaded in an ad-hoc manner like this, without any sort of
vulnerability scanning, etc.
As such, we should allow users to also package python packages in NiFi's native
archiving format (NARs).
The package structure should be as follows:
{code:java}
my-nar.nar
+-- META-INF/
+-- MANIFEST.MF
+-- NAR-INF/
+-- bundled-dependencies/
+-- dependency1
+-- dependency2
+-- etc.
+-- MyProcessor.py{code}
Where {{MyProcessor.py}} could also be a python module / directory.
In this way, we allow a Python Processor to be packaged up with its third party
dependencies and dropped in the lib/ directory (or extensions) directory of a
NiFi installation in the same way that a Java processor would be.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) ** Counter table has extra unnecessary can modfiy check * Bulletin Board (NIFI-12560) ** When the bulletin board polls it is missing the offset "after" query parameter * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) ** Clustering (NIFI-12807) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry (NIFI-12734) * Parameter Providers (NIFI-12622) ** Fetch parameters from provider, map to parameter context (dialog) - (NIFI-12665) * Cluster ** Status History - node specific values (NIFI-12848) * Flow Configuration History (NIFI-12754) ** ActionEntity.action should be optional * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) ** Don't show users or groups in create/edit dialog is there are none * Policies (NIFI-12548) ** Overridden policy Empty or Copy (NIFI-12679) ** Select Empty by default * Help (NIFI-12795) * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) ** Clustering (NIFI-12807) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) ** Clustering * Change Component Version * Consider PG permissions in Toolbox (NIFI-12683) * Handle linking to components that are not on the canvas * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs ** Disable and Configure * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Stop and Configure ** Open Custom UI (NIFI-12958) ** Property History ** Unable to re-add any removed Property (NIFI-12743) ** Shift-Enter new line when editing Property (NIFI-12743) * Property Verification * More Details (Processor, Controller Service, Reporting Task) * Download Flow * Create RPG (NIFI-12758) * Configure RPG (NIFI-12774) * RPG Remote Ports (NIFI-12778) * RPG Go To (NIFI-12759) * RPG Refresh (NIFI-12761) * Color * Move to Front * Copy/Paste * Add/Update Info Icons in dialogs throughout the application * Set viewport earlier when loading a Process Group (NIFI-12737) * Canvas global menu item should navigate user back to where they were on the canvas (NIFI-12737) * Better theme support (NIFI-12655) * Set up development/production environments files * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmation * Update API calls to use uiOnly flag * Use polling interval from API * Load FlowConfiguration in guard * Routing error handling * General API response error handling ** Management CS (NIFI-12663) ** Canvas CS (NIFI-12684) ** Remainder of Settings (NIFI-12723) ** Counters (NIFI-12723) ** Bulletins (NIFI-12723) ** Flow Designer ** Parameter Contexts (NIFI-12937) ** Provenance (NIFI-12767) ** Queue Listing (NIFI-12742) ** Summary (NIFI-12742) ** Users (NIFI-12742) ** Policies ** Status History * Introduce header in new pages to unify with canvas and offer better navigation. (NIFI-12597) * Theme docs, view flow file, and custom ui's * Prompt user to save Parameter Context when Edit form is dirty * Upgrade to Angular 17 (NIFI-12790) * Start/Stop processors, process groups, ... (NIFI-12568) * Dialog vertical resizing on smaller screens do not allow users to access all fields (NIFI-12603) * Flow Analysis report menu * Migrate NiFi UI to nx * Migrate from Karma tests to Jest
[jira] [Created] (NIFI-12958) Add support to open custom UIs
Matt Gilman created NIFI-12958:
--
Summary: Add support to open custom UIs
Key: NIFI-12958
URL: https://issues.apache.org/jira/browse/NIFI-12958
Project: Apache NiFi
Issue Type: Sub-task
Components: Core UI
Reporter: Matt Gilman
Assignee: Matt Gilman
Allow the user to open a custom UI.
1) Open Custom UI from context menu. Not from Advanced button in Configure
dialog.
2) Use route like the following...
{noformat}
/#/process-groups/43eb7385-018e-1000-2fb6-8dbbe88212cd/Processor/7bbab5aa-018e-1000-7de7-dc4052553b02/advanced{noformat}
3) Load component in question and using context path from component load UI in
iframe using appropriate values for it's query params.
4) New page should use the header.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-1872) Unify test utilities
[ https://issues.apache.org/jira/browse/MINIFICPP-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi reassigned MINIFICPP-1872: Assignee: Gábor Gyimesi > Unify test utilities > > > Key: MINIFICPP-1872 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1872 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > > The utilities used in our testing are split up in three different headers: > libminifi/test/Utils.h > libminifi/include/utils/IntegrationTestUtils.h > libminifi/include/utils/TestUtils.h > We should unify them in a single test utility file, also they should be part > of the test library under minifi::test namespace instead of the core minifi > lib. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12957 Upgrade Azure SDK BOM from 1.2.19 to 1.2.21 [nifi]
exceptionfactory opened a new pull request, #8571: URL: https://github.com/apache/nifi/pull/8571 # Summary [NIFI-12957](https://issues.apache.org/jira/browse/NIFI-12957) Upgrades the Azure SDK BOM from 1.2.19 to [1.2.21](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/boms/azure-sdk-bom/CHANGELOG.md) incorporating recent dependencies. The upgrade also removes the need for several custom overrides now that the transitive dependency versions have also been upgraded. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12957) Upgrade Azure SDK BOM to 1.2.21
[ https://issues.apache.org/jira/browse/NIFI-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12957: Status: Patch Available (was: Open) > Upgrade Azure SDK BOM to 1.2.21 > --- > > Key: NIFI-12957 > URL: https://issues.apache.org/jira/browse/NIFI-12957 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > Time Spent: 10m > Remaining Estimate: 0h > > The Azure SDK BOM dependency should be upgraded to > [1.2.21|https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/boms/azure-sdk-bom/CHANGELOG.md] > to incorporate the latest set of Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12957) Upgrade Azure SDK BOM to 1.2.21
[ https://issues.apache.org/jira/browse/NIFI-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12957: Description: The Azure SDK BOM dependency should be upgraded to [1.2.21|https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/boms/azure-sdk-bom/CHANGELOG.md] to incorporate the latest set of Azure libraries. (was: The Azure SDK BOM dependency should be upgraded to [1.2.21]([https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/boms/azure-sdk-bom/CHANGELOG.md)] to incorporate the latest set of Azure libraries.) > Upgrade Azure SDK BOM to 1.2.21 > --- > > Key: NIFI-12957 > URL: https://issues.apache.org/jira/browse/NIFI-12957 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > > The Azure SDK BOM dependency should be upgraded to > [1.2.21|https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/boms/azure-sdk-bom/CHANGELOG.md] > to incorporate the latest set of Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12957) Upgrade Azure SDK BOM to 1.2.21
David Handermann created NIFI-12957: --- Summary: Upgrade Azure SDK BOM to 1.2.21 Key: NIFI-12957 URL: https://issues.apache.org/jira/browse/NIFI-12957 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann The Azure SDK BOM dependency should be upgraded to [1.2.21]([https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/boms/azure-sdk-bom/CHANGELOG.md)] to incorporate the latest set of Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12957) Upgrade Azure SDK BOM to 1.2.21
[ https://issues.apache.org/jira/browse/NIFI-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12957: Labels: backport-needed (was: ) > Upgrade Azure SDK BOM to 1.2.21 > --- > > Key: NIFI-12957 > URL: https://issues.apache.org/jira/browse/NIFI-12957 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > > The Azure SDK BOM dependency should be upgraded to > [1.2.21]([https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/boms/azure-sdk-bom/CHANGELOG.md)] > to incorporate the latest set of Azure libraries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12857 Add test for and refactor implementation of QueuePrioritizer [nifi]
EndzeitBegins commented on PR #8466: URL: https://github.com/apache/nifi/pull/8466#issuecomment-2020987603 Of course, just rebased on the last commit of `main`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12900 Avoid unnecessary directory listing in PutSFTP [nifi]
EndzeitBegins commented on PR #8503: URL: https://github.com/apache/nifi/pull/8503#issuecomment-2020975403 Perfect. I applied your changes regarding the comment removal as is. As always, thank you for your review @exceptionfactory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2277 Add virtualenv support for python processors [nifi-minifi-cpp]
lordgamez commented on code in PR #1721:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1721#discussion_r1539722380
##
extensions/python/PythonDependencyInstaller.h:
##
@@ -0,0 +1,45 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+
+#include "PythonConfigState.h"
+#include "core/logging/Logger.h"
+#include "core/logging/LoggerConfiguration.h"
+#include "properties/Configure.h"
+
+namespace org::apache::nifi::minifi::extensions::python {
+
+class PythonDependencyInstaller {
+ public:
+ explicit PythonDependencyInstaller(const std::shared_ptr
);
+ void installDependenciesFromRequirementsFiles() const;
+
+ private:
+ std::vector getRequirementsFilePaths() const;
+ void createVirtualEnvIfSpecified() const;
+ static void evalScript(std::string_view script);
+ void addVirtualenvToPath() const;
+
+ PythonConfigState config_state_;
Review Comment:
Updated in 99359a0eda994fd5efa3095ebb76811ff1136f05
##
extensions/python/PythonDependencyInstaller.cpp:
##
@@ -0,0 +1,167 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "PythonDependencyInstaller.h"
+
+#include "PythonScriptException.h"
+#include "PythonInterpreter.h"
+#include "PyException.h"
+#include "types/Types.h"
+
+namespace org::apache::nifi::minifi::extensions::python {
+
+namespace {
+
+std::string getPythonBinary(const std::shared_ptr ) {
+#if WIN32
+ std::string python_binary = "python";
+#else
+ std::string python_binary = "python3";
+#endif
+ if (auto binary =
configuration->get(minifi::Configuration::nifi_python_env_setup_binary)) {
+python_binary = *binary;
+ }
+ return python_binary;
+}
+
+// On Windows when calling a system command using std::system, the whole
command needs to be encapsulated in additional quotes,
+// due to the std::system passing the command to 'cmd.exe /C' which needs the
additional quotes to handle the command as a single argument
+std::string encapsulateCommandInQuotesIfNeeded(const std::string& command) {
+#if WIN32
+return "\"" + command + "\"";
+#else
+return command;
+#endif
+}
+
+} // namespace
+
+PythonDependencyInstaller::PythonDependencyInstaller(const
std::shared_ptr ) {
+ config_state_.python_binary = getPythonBinary(configuration);
+ std::string automatic_install_str;
+ config_state_.install_python_packages_automatically =
+
configuration->get(Configuration::nifi_python_install_packages_automatically,
automatic_install_str) &&
utils::string::toBool(automatic_install_str).value_or(false);
+ if (auto path =
configuration->get(minifi::Configuration::nifi_python_virtualenv_directory)) {
+config_state_.virtualenv_path = *path;
+logger_->log_debug("Python virtualenv path was specified at: {}",
config_state_.virtualenv_path.string());
+ } else {
+logger_->log_debug("No valid python virtualenv path was specified");
+ }
+ if (auto python_processor_dir =
configuration->get(minifi::Configuration::nifi_python_processor_dir)) {
+config_state_.python_processor_dir = *python_processor_dir;
+logger_->log_debug("Python processor dir was specified at: {}",
config_state_.python_processor_dir.string());
+ } else {
+logger_->log_debug("No valid python processor dir was not specified in
properties");
+ }
+ createVirtualEnvIfSpecified();
+ addVirtualenvToPath();
+}
+
+std::vector
PythonDependencyInstaller::getRequirementsFilePaths() const {
+
Re: [PR] MINIFICPP-2277 Add virtualenv support for python processors [nifi-minifi-cpp]
lordgamez commented on code in PR #1721:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1721#discussion_r1539723152
##
extensions/python/PythonScriptEngine.h:
##
@@ -84,7 +48,7 @@ class PythonScriptEngine {
PythonScriptEngine& operator=(const PythonScriptEngine& other) = delete;
PythonScriptEngine& operator=(PythonScriptEngine&& other) = delete;
- static void initialize() {}
+ static void initialize(const std::shared_ptr ,
const std::shared_ptr& logger);
Review Comment:
Good catch, it was removed, but the declaration remained, removed in
99359a0eda994fd5efa3095ebb76811ff1136f05
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12900 Avoid unnecessary directory listing in PutSFTP [nifi]
exceptionfactory commented on code in PR #8503:
URL: https://github.com/apache/nifi/pull/8503#discussion_r1539559603
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/SFTPTransfer.java:
##
@@ -694,20 +708,12 @@ public FileInfo getRemoteFileInfo(final FlowFile
flowFile, final String path, St
}
}
-RemoteResourceInfo matchingEntry = null;
-for (final RemoteResourceInfo entry : remoteResources) {
-if (entry.getName().equalsIgnoreCase(filename)) {
-matchingEntry = entry;
-break;
-}
-}
-
// Previously JSCH would perform a listing on the full path (path +
filename) and would get an exception when it wasn't
// a file and then return null, so to preserve that behavior we return
null if the matchingEntry is a directory
Review Comment:
Given the change in behavior, these comments should be removed.
```suggestion
```
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/SFTPTransfer.java:
##
@@ -340,7 +356,7 @@ protected void getListing(final String path, final int
depth, final int maxResul
if (isIncludedFile(entry, symlink) && (filteringDisabled ||
pathMatched)) {
if (filteringDisabled || fileFilterPattern == null ||
fileFilterPattern.matcher(entryFilename).matches()) {
-listing.add(newFileInfo(entry, path));
+listing.add(newFileInfo(path, entry.getName(),
entry.getAttributes()));
Review Comment:
Regarding the second question, it appears that the behavior remains the
same. Thanks for the helpful reply, this looks good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12900 Avoid unnecessary directory listing in PutSFTP [nifi]
exceptionfactory commented on code in PR #8503:
URL: https://github.com/apache/nifi/pull/8503#discussion_r1539537745
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/SFTPTransfer.java:
##
@@ -340,7 +356,7 @@ protected void getListing(final String path, final int
depth, final int maxResul
if (isIncludedFile(entry, symlink) && (filteringDisabled ||
pathMatched)) {
if (filteringDisabled || fileFilterPattern == null ||
fileFilterPattern.matcher(entryFilename).matches()) {
-listing.add(newFileInfo(entry, path));
+listing.add(newFileInfo(path, entry.getName(),
entry.getAttributes()));
Review Comment:
Very good point @EndzeitBegins, on closer inspection, I agree with you that
the null check there should be unnecessary given that it would have failed in
`isIncludedFile()` based on current behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12900 Avoid unnecessary directory listing in PutSFTP [nifi]
EndzeitBegins commented on PR #8503: URL: https://github.com/apache/nifi/pull/8503#issuecomment-2020788649 I haven't encountered issues regarding symbolic links in the (arguably limited) manual testing I've done. However, I do not quite understand what you're addressing. The PR does not change the way `getListing` addresses the SFTP server. `stat` is only used when inquiring for a single resource using `getRemoteFileInfo`. The only change in `getListing` is how the results are mapped. However, even that's not really the case. Where before `newFileInfo` would receive a `RemoteResourceInfo` and access `.getAttributes()`, `.getName()`, and `.isDirectory()` it now receives the first two as arguments directly. The third one was just a utility function that accessed the same attributes under the hood which is basically inlined. I'm unsure how this would affect `getListing` to be honest. 樂 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2293 Support installing python dependencies defined inline [nifi-minifi-cpp]
lordgamez commented on code in PR #1727: URL: https://github.com/apache/nifi-minifi-cpp/pull/1727#discussion_r1539519179 ## docker/test/integration/cluster/ImageStore.py: ## @@ -97,10 +105,21 @@ def __build_minifi_cpp_sql_image(self): return self.__build_image(dockerfile) -def __build_minifi_cpp_image_with_nifi_python_processors(self, additional_cmd=""): +def __build_minifi_cpp_image_with_nifi_python_processors(self, python_option): parse_document_url = "https://raw.githubusercontent.com/apache/nifi/rel/nifi-; + NifiContainer.NIFI_VERSION + "/nifi-python-extensions/nifi-text-embeddings-module/src/main/python/ParseDocument.py" chunk_document_url = "https://raw.githubusercontent.com/apache/nifi/rel/nifi-; + NifiContainer.NIFI_VERSION + "/nifi-python-extensions/nifi-text-embeddings-module/src/main/python/ChunkDocument.py" pip3_install_command = "" +requirements_install_command = "" +additional_cmd = "" +parse_document_sed_cmd = 'sed -i "54d;55d" /opt/minifi/minifi-current/minifi-python/nifi_python_processors/ParseDocument.py && \\' +chunk_document_sed_cmd = 'sed -i "112d" /opt/minifi/minifi-current/minifi-python/nifi_python_processors/ChunkDocument.py && \\' +if python_option == PythonOptions.SYSTEM_INSTALLED_PACKAGES: +additional_cmd = "RUN pip3 install 'langchain<=0.17.0'" +elif python_option == PythonOptions.REQUIREMENTS_FILE: +requirements_install_command = "echo 'langchain<=0.17.0' > /opt/minifi/minifi-current/minifi-python/nifi_python_processors/requirements.txt && \\" +elif python_option == PythonOptions.INLINE_DEFINED_PACKAGES: +parse_document_sed_cmd = parse_document_sed_cmd[:-2] + ' sed -i "54 i \\ \\ \\ \\ \\ \\ \\ \\ dependencies = [\\"langchain<=0.17.0\\"]" /opt/minifi/minifi-current/minifi-python/nifi_python_processors/ParseDocument.py && \\' Review Comment: I think you are right, I tested and it also works so I updated it in 327ddca15c6bf2f9692d3da061f5688596537c06 ## extensions/python/pythonprocessors/nifi_python_processors/utils/inline_dependency_installer.py: ## @@ -0,0 +1,43 @@ +import ast +import sys +import subprocess +import os + + +class Visitor(ast.NodeVisitor): Review Comment: Good point, added some comments in 327ddca15c6bf2f9692d3da061f5688596537c06 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12956) Add semantic styles for the basic theme colors
Rob Fellows created NIFI-12956:
--
Summary: Add semantic styles for the basic theme colors
Key: NIFI-12956
URL: https://issues.apache.org/jira/browse/NIFI-12956
Project: Apache NiFi
Issue Type: Sub-task
Components: Core UI
Reporter: Rob Fellows
We should add themed styles for the commonly used colors from the palettes. I'm
thinking like the 3 (light, default, dark) main colors from each palette.
Something like this:
{code:java}
.warn-light {
color: $warn-palette-light
}
.warn-default {
color: $warn-palette-default
}
.warn-dark {
color: $warn-palette-dark
}
.primary-light {
color: $primary-palette-light
}
.primary-default {
color: $primary-palette-default
}
.primary-dark {
color: $primary-palette-dark
}
.accent-light {
color: $accent-palette-light
}
.accent-default {
color: $accent-palette-default
}
.accent-dark {
color: $accent-palette-dark
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) ** Counter table has extra unnecessary can modfiy check * Bulletin Board (NIFI-12560) ** When the bulletin board polls it is missing the offset "after" query parameter * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) ** Clustering (NIFI-12807) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry (NIFI-12734) * Parameter Providers (NIFI-12622) ** Fetch parameters from provider, map to parameter context (dialog) - (NIFI-12665) * Cluster ** Status History - node specific values (NIFI-12848) * Flow Configuration History (NIFI-12754) ** ActionEntity.action should be optional * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) ** Don't show users or groups in create/edit dialog is there are none * Policies (NIFI-12548) ** Overridden policy Empty or Copy (NIFI-12679) ** Select Empty by default * Help (NIFI-12795) * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) ** Clustering (NIFI-12807) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) ** Clustering * Change Component Version * Consider PG permissions in Toolbox (NIFI-12683) * Handle linking to components that are not on the canvas * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs ** Disable and Configure * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Stop and Configure ** Open Custom UI ** Property History ** Unable to re-add any removed Property (NIFI-12743) ** Shift-Enter new line when editing Property (NIFI-12743) * Property Verification * More Details (Processor, Controller Service, Reporting Task) * Download Flow * Create RPG (NIFI-12758) * Configure RPG (NIFI-12774) * RPG Remote Ports (NIFI-12778) * RPG Go To (NIFI-12759) * RPG Refresh (NIFI-12761) * Color * Move to Front * Copy/Paste * Add/Update Info Icons in dialogs throughout the application * Set viewport earlier when loading a Process Group (NIFI-12737) * Canvas global menu item should navigate user back to where they were on the canvas (NIFI-12737) * Better theme support (NIFI-12655) * Set up development/production environments files * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmation * Update API calls to use uiOnly flag * Use polling interval from API * Load FlowConfiguration in guard * Routing error handling * General API response error handling ** Management CS (NIFI-12663) ** Canvas CS (NIFI-12684) ** Remainder of Settings (NIFI-12723) ** Counters (NIFI-12723) ** Bulletins (NIFI-12723) ** Flow Designer ** Parameter Contexts (NIFI-12937) ** Provenance (NIFI-12767) ** Queue Listing (NIFI-12742) ** Summary (NIFI-12742) ** Users (NIFI-12742) ** Policies ** Status History * Introduce header in new pages to unify with canvas and offer better navigation. (NIFI-12597) * Theme docs, view flow file, and custom ui's * Prompt user to save Parameter Context when Edit form is dirty * Upgrade to Angular 17 (NIFI-12790) * Start/Stop processors, process groups, ... (NIFI-12568) * Dialog vertical resizing on smaller screens do not allow users to access all fields (NIFI-12603) * Flow Analysis report menu * Migrate NiFi UI to nx * Migrate from Karma tests to Jest (NIFI-12941) * Add
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) ** Counter table has extra unnecessary can modfiy check * Bulletin Board (NIFI-12560) ** When the bulletin board polls it is missing the offset "after" query parameter * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) ** Clustering (NIFI-12807) * Configure Reporting Task (NIFI-12563) * Flow Analysis Rules (NIFI-12588) * Registry Clients (NIFI-12486) * Import from Registry (NIFI-12734) * Parameter Providers (NIFI-12622) ** Fetch parameters from provider, map to parameter context (dialog) - (NIFI-12665) * Cluster ** Status History - node specific values (NIFI-12848) * Flow Configuration History (NIFI-12754) ** ActionEntity.action should be optional * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) ** Don't show users or groups in create/edit dialog is there are none * Policies (NIFI-12548) ** Overridden policy Empty or Copy (NIFI-12679) ** Select Empty by default * Help (NIFI-12795) * About * Show Upstream/Downstream * Align * List Queue (NIFI-12589) ** Clustering (NIFI-12807) * Empty [all] Queue (NIFI-12604) * View Content (NIFI-12589 and NIFI-12445) * View State (NIFI-12611) ** Clustering * Change Component Version * Consider PG permissions in Toolbox (NIFI-12683) * Handle linking to components that are not on the canvas * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs ** Disable and Configure * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) ** Stop and Configure ** Open Custom UI ** Property History ** Unable to re-add any removed Property (NIFI-12743) ** Shift-Enter new line when editing Property (NIFI-12743) * Property Verification * More Details (Processor, Controller Service, Reporting Task) * Download Flow * Create RPG (NIFI-12758) * Configure RPG (NIFI-12774) * RPG Remote Ports (NIFI-12778) * RPG Go To (NIFI-12759) * RPG Refresh (NIFI-12761) * Color * Move to Front * Copy/Paste * Add/Update Info Icons in dialogs throughout the application * Set viewport earlier when loading a Process Group (NIFI-12737) * Canvas global menu item should navigate user back to where they were on the canvas (NIFI-12737) * Better theme support (NIFI-12655) * Set up development/production environments files * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmation * Update API calls to use uiOnly flag * Use polling interval from API * Load FlowConfiguration in guard * Routing error handling * General API response error handling ** Management CS (NIFI-12663) ** Canvas CS (NIFI-12684) ** Remainder of Settings (NIFI-12723) ** Counters (NIFI-12723) ** Bulletins (NIFI-12723) ** Flow Designer ** Parameter Contexts (NIFI-12937) ** Provenance (NIFI-12767) ** Queue Listing (NIFI-12742) ** Summary (NIFI-12742) ** Users (NIFI-12742) ** Policies ** Status History * Introduce header in new pages to unify with canvas and offer better navigation. (NIFI-12597) * Theme docs, view flow file, and custom ui's * Prompt user to save Parameter Context when Edit form is dirty * Upgrade to Angular 17 (NIFI-12790) * Start/Stop processors, process groups, ... (NIFI-12568) * Dialog vertical resizing on smaller screens do not allow users to access all fields (NIFI-12603) * Flow Analysis report menu * Migrate NiFi UI to nx * Migrate from Karma tests to Jest (NIFI-12941) * Add
Re: [PR] NIFI-12937: Adding error handling to the Parameter Context Listing page [nifi]
rfellows commented on code in PR #8555:
URL: https://github.com/apache/nifi/pull/8555#discussion_r1539488996
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/parameter-contexts/ui/parameter-context-listing/edit-parameter-context/edit-parameter-context.component.html:
##
@@ -26,12 +27,21 @@ {{ this.isNew ? 'Add' : 'Edit' }}
Parameter Context
@for (updateStep of requestEntity.request.updateSteps;
track updateStep) {
{{ updateStep.description
}}
-@if (updateStep.complete) {
-
+@if (updateStep.failureReason) {
+
Review Comment:
This should be using a class that derives its color from the theming system.
It should probably be the default color from the warn palette.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/parameter-contexts/state/parameter-context-listing/parameter-context-listing.reducer.ts:
##
@@ -74,10 +73,15 @@ export const parameterContextListingReducer = createReducer(
...state,
saving: true
})),
-on(submitParameterContextUpdateRequestSuccess,
pollParameterContextUpdateRequestSuccess, (state, { response }) => ({
-...state,
-updateRequestEntity: response.requestEntity
-})),
+on(
+submitParameterContextUpdateRequestSuccess,
+pollParameterContextUpdateRequestSuccess,
+deleteParameterContextUpdateRequestSuccess,
+(state, { response }) => ({
Review Comment:
We should probably be setting `saving: false` here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12888) OAuth2 token refresh doesn't work in Email processors
[ https://issues.apache.org/jira/browse/NIFI-12888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830985#comment-17830985 ] ASF subversion and git services commented on NIFI-12888: Commit bb52b839a4ff0425fa0d65c517fc7ce7441e264f in nifi's branch refs/heads/support/nifi-1.x from Tamas Palfy [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bb52b839a4 ] NIFI-12888 In AbstractEmailProcessor check for expired oauth2 token when determining whether mail receiver needs to be recreated. Signed-off-by: Pierre Villard This closes #8494. > OAuth2 token refresh doesn't work in Email processors > - > > Key: NIFI-12888 > URL: https://issues.apache.org/jira/browse/NIFI-12888 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > ConsumeIMAP and ConsumePOP3 both inherit the AbstractEmailProcessor in which > the OAuth2 token refresh handling is not handled properly. > The object that is receiving the emails is created and finalized only once. > When the token expires, nothing makes to refresh logic to run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12888) OAuth2 token refresh doesn't work in Email processors
[ https://issues.apache.org/jira/browse/NIFI-12888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-12888. --- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed > OAuth2 token refresh doesn't work in Email processors > - > > Key: NIFI-12888 > URL: https://issues.apache.org/jira/browse/NIFI-12888 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 1h > Remaining Estimate: 0h > > ConsumeIMAP and ConsumePOP3 both inherit the AbstractEmailProcessor in which > the OAuth2 token refresh handling is not handled properly. > The object that is receiving the emails is created and finalized only once. > When the token expires, nothing makes to refresh logic to run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12900 Avoid unnecessary directory listing in PutSFTP [nifi]
EndzeitBegins commented on code in PR #8503:
URL: https://github.com/apache/nifi/pull/8503#discussion_r1539482164
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/SFTPTransfer.java:
##
@@ -340,7 +356,7 @@ protected void getListing(final String path, final int
depth, final int maxResul
if (isIncludedFile(entry, symlink) && (filteringDisabled ||
pathMatched)) {
if (filteringDisabled || fileFilterPattern == null ||
fileFilterPattern.matcher(entryFilename).matches()) {
-listing.add(newFileInfo(entry, path));
+listing.add(newFileInfo(path, entry.getName(),
entry.getAttributes()));
Review Comment:
Thank you for your feedback @exceptionfactory.
The call to `newFileInfo` is preceded by a call to `isIncludedFile` inside
the if expression, which in turn calls `.isRegularFile()` on the
`RemoteResourceInfo entry` passed in.
In case the `entry` was `null`, a `NullPointerException` would be thrown
before reaching `newFileInfo`.
Thus I figured the null check is not required in `newFileInfo`, but I can
add an explicit `null` check into the if clause, if that's desired.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12888) OAuth2 token refresh doesn't work in Email processors
[ https://issues.apache.org/jira/browse/NIFI-12888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830984#comment-17830984 ] ASF subversion and git services commented on NIFI-12888: Commit 92830af2e986dafe243f770bd672e6d969070746 in nifi's branch refs/heads/main from Tamas Palfy [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=92830af2e9 ] NIFI-12888 In AbstractEmailProcessor check for expired oauth2 token when determining whether mail receiver needs to be recreated. Signed-off-by: Pierre Villard This closes #8494. > OAuth2 token refresh doesn't work in Email processors > - > > Key: NIFI-12888 > URL: https://issues.apache.org/jira/browse/NIFI-12888 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > ConsumeIMAP and ConsumePOP3 both inherit the AbstractEmailProcessor in which > the OAuth2 token refresh handling is not handled properly. > The object that is receiving the emails is created and finalized only once. > When the token expires, nothing makes to refresh logic to run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12888 In AbstractEmailProcessor recreate email receiver when token expires. [nifi]
asfgit closed pull request #8494: NIFI-12888 In AbstractEmailProcessor recreate email receiver when token expires. URL: https://github.com/apache/nifi/pull/8494 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830969#comment-17830969 ] Daniel Stieglitz commented on NIFI-11167: - [~iiojj2] Can you please make a separate ticket for this? Thanks! > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0-M1, 1.23.0 > > Attachments: CSVRecordSetWriter_configuration.png, > ExcelReaderConfiguration.png, QueryRecord_configuration.png, Test > ExcelReader.xlsx, Test Sheet-formula.xlsx, image-2023-11-28-18-22-07-446.png, > image-2023-11-29-15-51-08-386.png, resulting.csv, screenshot-1.png > > Time Spent: 10h 10m > Remaining Estimate: 0h > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12888 In AbstractEmailProcessor recreate email receiver when token expires. [nifi]
dan-s1 commented on PR #8494: URL: https://github.com/apache/nifi/pull/8494#issuecomment-2020630980 @exceptionfactory I think with the latest commit everything seems okay. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2293 Support installing python dependencies defined inline [nifi-minifi-cpp]
lordgamez commented on code in PR #1727:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1727#discussion_r1539359322
##
extensions/python/pythonprocessors/nifi_python_processors/utils/inline_dependency_installer.py:
##
@@ -0,0 +1,43 @@
+import ast
+import sys
+import subprocess
+import os
+
+
+class Visitor(ast.NodeVisitor):
+def __init__(self, class_name):
+self.dependencies = []
+self.class_name = class_name
+
+def visit_ClassDef(self, node):
+if node.name != self.class_name:
+return
+for child in node.body:
+if isinstance(child, ast.ClassDef) and child.name ==
'ProcessorDetails':
+for detail in child.body:
+if isinstance(detail, ast.Assign) and detail.targets[0].id
== 'dependencies':
+for elt in detail.value.elts:
+if isinstance(elt, ast.Constant):
+self.dependencies.append(elt.s)
+break
+break
+
+
+def extract_dependencies(file_path):
+class_name = file_path.split(os.sep)[-1].split('.')[0]
+with open(file_path, 'r') as file:
+code = file.read()
+
+tree = ast.parse(code)
+visitor = Visitor(class_name)
+visitor.visit(tree)
+return visitor.dependencies
+
+
+if __name__ == '__main__':
+if len(sys.argv) < 2:
+sys.exit(1)
+
+dependencies = extract_dependencies(sys.argv[1])
+if dependencies:
+subprocess.check_call([sys.executable, "-m", "pip", "install",
"--no-cache-dir"] + dependencies)
Review Comment:
I only used it because NiFi also uses it in its pip installs for the python
processors, probably to avoid any issues from previous pip installs. I don't
think it's that necessary, but I thought better to follow NiFi's pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12955) Update Dependency Check Suppressions
[ https://issues.apache.org/jira/browse/NIFI-12955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12955: Status: Patch Available (was: Open) > Update Dependency Check Suppressions > > > Key: NIFI-12955 > URL: https://issues.apache.org/jira/browse/NIFI-12955 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The OWASP Dependency Check suppressions configuration includes a number of > items that no longer apply based on library upgrades and improvements to the > scanner itself. Unused suppressions should be removed and others should be > added as needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2309 Add JSON flow config examples [nifi-minifi-cpp]
lordgamez commented on code in PR #1740:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1740#discussion_r1539352073
##
examples/tailfile_config.json:
##
@@ -0,0 +1,51 @@
+{
+"rootGroup": {
+"name": "MiNiFi Flow",
+"processors": [
+{
+"name": "Log tailfile attributes",
+"identifier": "41141af1-3c43-4fa4-8236-2b207b0fe9d8",
+"type": "org.apache.nifi.minifi.processors.LogAttribute",
+"schedulingStrategy": "TIMER_DRIVEN",
+"schedulingPeriod": "1000 ms",
Review Comment:
I think it should be event driven as we recommend using that in this case,
there are other occurrences of timer driven cases. Updated in
d67362d317cc2369536a9eede32709da58541444
##
examples/README.md:
##
@@ -37,122 +43,158 @@ The following examples show simple flow configurations
for several common use ca
- [Upload Blob to Azure Storage](#upload-blob-to-azure-storage)
- [Put Object in AWS S3 Bucket](#put-object-in-aws-s3-bucket)
- [List and Fetch Content from AWS S3
Bucket](#list-and-fetch-content-from-aws-s3-bucket)
+ - [Merge, compress and upload files to Google Cloud
Storage](#merge-compress-and-upload-files-to-google-cloud-storage)
- [SQL Operations](#sql-operations)
- [Query Database Table](#query-database-table)
-- [ExecuteScript](#ExecuteScript)
- - [Reverse flowfile's content with lua](#reverse-content-with-lua)
- - [Reverse flowfile's content with python](#reverse-content-with-python)
+- [ExecuteScript](#executescript)
+ - [Reverse Content with Scripts](#reverse-content-with-scripts)
+- [Grafana Loki](#grafana-loki)
+ - [Send logs to Grafana Loki](#send-logs-to-grafana-loki)
+- [MQTT Operations](#mqtt-operations)
+ - [Publish Message to MQTT Broker](#publish-message-to-mqtt-broker)
+- [Network Operations](#network-operations)
+ - [Send Data to Remote Host through
TCP](#send-data-to-remote-host-through-tcp)
## Filesystem Operations
### Getting File Data and Putting It in an Output Directory
-Using the [getfile_putfile_config.yml](getfile_putfile_config.yml) flow
configuration MiNiFi gets all files of minimum 1MB size from the
`/tmp/getfile_dir` directory and puts them in the `/tmp/out_dir` output
directory.
+Using the
[getfile_putfile_config.yml](getfile_putfile_config.yml)/[getfile_putfile_config.json](getfile_putfile_config.json)
flow configuration MiNiFi gets all files of minimum 1MB size from the
`/tmp/getfile_dir` directory and puts them in the `/tmp/out_dir` output
directory.
-The flow: GetFile success PutFile
+The flow: [GetFile](../PROCESSORS.md#getfile)
[PutFile](../PROCESSORS.md#putfile)
### Tailing a Single File
-Using the [tailfile_config.yml](tailfile_config.yml) flow configuration MiNiFi
tails a single file `/tmp/test_file.log` and creates flowfiles from every
single line, then logs the flowfile attributes.
+Using the
[tailfile_config.yml](tailfile_config.yml)/[tailfile_config.json](tailfile_config.json)
flow configuration MiNiFi tails a single file `/tmp/test_file.log` and creates
flowfiles from every single line, then logs the flowfile attributes.
-The flow: TailFile success LogAttribute
+The flow: [TailFile](../PROCESSORS.md#tailfile)
[LogAttribute](../PROCESSORS.md#logattribute)
## Windows Specific Processors
### Consuming Windows Event Logs
-Using the [cwel_config.yml](cwel_config.yml) flow configuration MiNiFi queries
all Windows system events and puts them to the `C:\temp\` directory in
flattened JSON format.
+Using the
[cwel_config.yml](cwel_config.yml)/[cwel_config.json](cwel_config.json) flow
configuration MiNiFi queries all Windows system events and puts them to the
`C:\temp\` directory in flattened JSON format.
-The flow: ConsumeWindowsEventLog PutFile
+The flow: [ConsumeWindowsEventLog](../PROCESSORS.md#consumewindowseventlog)
[PutFile](../PROCESSORS.md#putfile)
### Reading System Performance Data
-Using the [pdh_config.yml](pdh_config.yml) flow configuration MiNiFi reads CPU
and Disk performance data through Windows' Performance Data Helper (PDH)
component and puts the data to the `C:\temp\` directory in a compact JSON
format.
+Using the [pdh_config.yml](pdh_config.yml)/[pdh_config.json](pdh_config.json)
flow configuration MiNiFi reads CPU and Disk performance data through Windows'
Performance Data Helper (PDH) component and puts the data to the `C:\temp\`
directory in a compact JSON format.
-The flow: PerformanceDataMonitor PutFile
+The flow: [PerformanceDataMonitor](../PROCESSORS.md#performancedatamonitor)
[PutFile](../PROCESSORS.md#putfile)
## Linux Specific Processors
### Consume Systemd-Journald System Journal Messages
-Using the [consumejournald_config.yml](consumejournald_config.yml) flow
configuration MiNiFi reads systemd-journald journal messages and logs them on
`info` level.
+Using the
[PR] NIFI-12955 Update OWASP Dependency Check Suppressions [nifi]
exceptionfactory opened a new pull request, #8570: URL: https://github.com/apache/nifi/pull/8570 # Summary [NIFI-12955](https://issues.apache.org/jira/browse/NIFI-12955) Updates the OWASP Dependency Check suppressions configuration, removing a number of unused suppressions that are no longer necessary as a result of direct library upgrades and improvements in the scanner itself. New suppressions include Clojure spec libraries that have a different version number than Clojure itself, and the shaded Protobuf library from Hadoop. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12954) Upgrade AWS BOM references to 2.25.16 and 1.12.686
[ https://issues.apache.org/jira/browse/NIFI-12954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12954: Summary: Upgrade AWS BOM references to 2.25.16 and 1.12.686 (was: Upgrade AWS BOM references) > Upgrade AWS BOM references to 2.25.16 and 1.12.686 > -- > > Key: NIFI-12954 > URL: https://issues.apache.org/jira/browse/NIFI-12954 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade AWS BOM references > * 1.x from 1.12.637 to 1.12.686 > * 2.x from 2.23.3 to 2.25.16 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12953) Upgrade AWS Kinesis client to 2.5.7
[ https://issues.apache.org/jira/browse/NIFI-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12953: Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade AWS Kinesis client to 2.5.7 > --- > > Key: NIFI-12953 > URL: https://issues.apache.org/jira/browse/NIFI-12953 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade AWS Kinesis client to 2.5.7 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12954) Upgrade AWS BOM references to 2.25.16 and 1.12.686
[ https://issues.apache.org/jira/browse/NIFI-12954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12954: Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade AWS BOM references to 2.25.16 and 1.12.686 > -- > > Key: NIFI-12954 > URL: https://issues.apache.org/jira/browse/NIFI-12954 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade AWS BOM references > * 1.x from 1.12.637 to 1.12.686 > * 2.x from 2.23.3 to 2.25.16 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12954) Upgrade AWS BOM references
[ https://issues.apache.org/jira/browse/NIFI-12954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830949#comment-17830949 ] ASF subversion and git services commented on NIFI-12954: Commit b16a4c12329c4a47a3553b71b99bcd23a85a1fdf in nifi's branch refs/heads/support/nifi-1.x from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b16a4c1232 ] NIFI-12954 Upgraded AWS BOM from 2.23.3 to 2.25.16 - Upgraded AWS BOM from 1.12.637 to 1.12.686 This closes #8569 Signed-off-by: David Handermann (cherry picked from commit 0e255674fdc175dd14e8728420ed0c8d8fb1326b) > Upgrade AWS BOM references > -- > > Key: NIFI-12954 > URL: https://issues.apache.org/jira/browse/NIFI-12954 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade AWS BOM references > * 1.x from 1.12.637 to 1.12.686 > * 2.x from 2.23.3 to 2.25.16 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12953) Upgrade AWS Kinesis client to 2.5.7
[ https://issues.apache.org/jira/browse/NIFI-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830944#comment-17830944 ] ASF subversion and git services commented on NIFI-12953: Commit b1f9df5091cf4795c6b14969074a7eb9075e95cf in nifi's branch refs/heads/main from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b1f9df5091 ] NIFI-12953 Upgraded AWS Kinesis client from 2.5.3 to 2.5.7 This closes #8568 Signed-off-by: David Handermann > Upgrade AWS Kinesis client to 2.5.7 > --- > > Key: NIFI-12953 > URL: https://issues.apache.org/jira/browse/NIFI-12953 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade AWS Kinesis client to 2.5.7 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12954) Upgrade AWS BOM references
[ https://issues.apache.org/jira/browse/NIFI-12954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830945#comment-17830945 ] ASF subversion and git services commented on NIFI-12954: Commit 0e255674fdc175dd14e8728420ed0c8d8fb1326b in nifi's branch refs/heads/main from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0e255674fd ] NIFI-12954 Upgraded AWS BOM from 2.23.3 to 2.25.16 - Upgraded AWS BOM from 1.12.637 to 1.12.686 This closes #8569 Signed-off-by: David Handermann > Upgrade AWS BOM references > -- > > Key: NIFI-12954 > URL: https://issues.apache.org/jira/browse/NIFI-12954 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade AWS BOM references > * 1.x from 1.12.637 to 1.12.686 > * 2.x from 2.23.3 to 2.25.16 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12954 - Upgrade AWS BOM references [nifi]
exceptionfactory closed pull request #8569: NIFI-12954 - Upgrade AWS BOM references URL: https://github.com/apache/nifi/pull/8569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12953 - Upgrade AWS Kinesis client to 2.5.7 [nifi]
exceptionfactory closed pull request #8568: NIFI-12953 - Upgrade AWS Kinesis client to 2.5.7 URL: https://github.com/apache/nifi/pull/8568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12955) Update Dependency Check Suppressions
David Handermann created NIFI-12955: --- Summary: Update Dependency Check Suppressions Key: NIFI-12955 URL: https://issues.apache.org/jira/browse/NIFI-12955 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Reporter: David Handermann Assignee: David Handermann The OWASP Dependency Check suppressions configuration includes a number of items that no longer apply based on library upgrades and improvements to the scanner itself. Unused suppressions should be removed and others should be added as needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830940#comment-17830940 ] Philipp Korniets commented on NIFI-11167: - another thing for ExcelReader - passowrd protected xlsx. [~dstiegli1] > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0-M1, 1.23.0 > > Attachments: CSVRecordSetWriter_configuration.png, > ExcelReaderConfiguration.png, QueryRecord_configuration.png, Test > ExcelReader.xlsx, Test Sheet-formula.xlsx, image-2023-11-28-18-22-07-446.png, > image-2023-11-29-15-51-08-386.png, resulting.csv, screenshot-1.png > > Time Spent: 10h 10m > Remaining Estimate: 0h > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12941] - Migrate to Jest from Karma/Jasmine. [nifi]
mcgilman commented on PR #8566: URL: https://github.com/apache/nifi/pull/8566#issuecomment-2020331868 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12954 - Upgrade AWS BOM references [nifi]
pvillard31 opened a new pull request, #8569: URL: https://github.com/apache/nifi/pull/8569 # Summary [NIFI-12954](https://issues.apache.org/jira/browse/NIFI-12954) - Upgrade AWS BOM references - https://github.com/aws/aws-sdk-java-v2/blob/master/CHANGELOG.md - https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12954) Upgrade AWS BOM references
[ https://issues.apache.org/jira/browse/NIFI-12954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12954: -- Status: Patch Available (was: Open) > Upgrade AWS BOM references > -- > > Key: NIFI-12954 > URL: https://issues.apache.org/jira/browse/NIFI-12954 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > Upgrade AWS BOM references > * 1.x from 1.12.637 to 1.12.686 > * 2.x from 2.23.3 to 2.25.16 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12954) Upgrade AWS BOM references
Pierre Villard created NIFI-12954: - Summary: Upgrade AWS BOM references Key: NIFI-12954 URL: https://issues.apache.org/jira/browse/NIFI-12954 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Assignee: Pierre Villard Upgrade AWS BOM references * 1.x from 1.12.637 to 1.12.686 * 2.x from 2.23.3 to 2.25.16 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12953) Upgrade AWS Kinesis client to 2.5.7
[ https://issues.apache.org/jira/browse/NIFI-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12953: -- Status: Patch Available (was: Open) > Upgrade AWS Kinesis client to 2.5.7 > --- > > Key: NIFI-12953 > URL: https://issues.apache.org/jira/browse/NIFI-12953 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade AWS Kinesis client to 2.5.7 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12953 - Upgrade AWS Kinesis client to 2.5.7 [nifi]
pvillard31 opened a new pull request, #8568: URL: https://github.com/apache/nifi/pull/8568 # Summary [NIFI-12953](https://issues.apache.org/jira/browse/NIFI-12953) - Upgrade AWS Kinesis client to 2.5.7 - https://github.com/google/guava/releases - https://github.com/awslabs/amazon-kinesis-client/blob/master/CHANGELOG.md # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12939: Retry Kerberos login on authentication failure in Iceberg processors [nifi]
mark-bathori commented on code in PR #8553:
URL: https://github.com/apache/nifi/pull/8553#discussion_r1539099254
##
nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-processors/src/main/java/org/apache/nifi/processors/iceberg/PutIceberg.java:
##
@@ -299,16 +309,23 @@ public void doOnTrigger(ProcessContext context,
ProcessSession session, FlowFile
final WriteResult result = taskWriter.complete();
appendDataFiles(context, flowFile, table, result);
} catch (Exception e) {
-getLogger().error("Exception occurred while writing iceberg
records. Removing uncommitted data files", e);
+final Optional causeOptional = findCause(e,
GSSException.class, gsse -> GSSException.NO_CRED == gsse.getMajor());
+if (causeOptional.isPresent()) {
+getLogger().warn("No valid Kerberos credential found, retrying
login", causeOptional.get());
+initKerberosCredentials(context);
+session.rollback(true);
Review Comment:
Thanks @mattyb149 for the review. Yes you are right, yielding in case of
this authentication error makes more sense than penalize. I'll update this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-12953) Upgrade AWS Kinesis client to 2.5.7
Pierre Villard created NIFI-12953: - Summary: Upgrade AWS Kinesis client to 2.5.7 Key: NIFI-12953 URL: https://issues.apache.org/jira/browse/NIFI-12953 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Assignee: Pierre Villard Upgrade AWS Kinesis client to 2.5.7 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12937: Adding error handling to the Parameter Context Listing page [nifi]
rfellows commented on PR #8555: URL: https://github.com/apache/nifi/pull/8555#issuecomment-2020246850 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12949) Upgrade Box SDK to 4.8.0
[ https://issues.apache.org/jira/browse/NIFI-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830870#comment-17830870 ] ASF subversion and git services commented on NIFI-12949: Commit bdbcf30e5dedd8ff89acf726847ecc4b972f596e in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bdbcf30e5d ] NIFI-12949 Upgraded Box SDK from 4.6.1 to 4.8.0 Signed-off-by: Pierre Villard This closes #8567. > Upgrade Box SDK to 4.8.0 > > > Key: NIFI-12949 > URL: https://issues.apache.org/jira/browse/NIFI-12949 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Time Spent: 20m > Remaining Estimate: 0h > > The Box Java SDK should be upgraded to > [4.8.0|https://github.com/box/box-java-sdk/releases/tag/v4.8.0] to > incorporate several transitive dependency upgrades, including more recent > Bouncy Castle dependency coordinates. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12949) Upgrade Box SDK to 4.8.0
[ https://issues.apache.org/jira/browse/NIFI-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12949: -- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Box SDK to 4.8.0 > > > Key: NIFI-12949 > URL: https://issues.apache.org/jira/browse/NIFI-12949 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The Box Java SDK should be upgraded to > [4.8.0|https://github.com/box/box-java-sdk/releases/tag/v4.8.0] to > incorporate several transitive dependency upgrades, including more recent > Bouncy Castle dependency coordinates. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12949) Upgrade Box SDK to 4.8.0
[ https://issues.apache.org/jira/browse/NIFI-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830869#comment-17830869 ] ASF subversion and git services commented on NIFI-12949: Commit ebac490715415763f5a4f704d37e631cd2f6a5b8 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ebac490715 ] NIFI-12949 Upgraded Box SDK from 4.6.1 to 4.8.0 Signed-off-by: Pierre Villard This closes #8567. > Upgrade Box SDK to 4.8.0 > > > Key: NIFI-12949 > URL: https://issues.apache.org/jira/browse/NIFI-12949 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Time Spent: 10m > Remaining Estimate: 0h > > The Box Java SDK should be upgraded to > [4.8.0|https://github.com/box/box-java-sdk/releases/tag/v4.8.0] to > incorporate several transitive dependency upgrades, including more recent > Bouncy Castle dependency coordinates. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12949 Upgrade Box SDK from 4.6.1 to 4.8.0 [nifi]
asfgit closed pull request #8567: NIFI-12949 Upgrade Box SDK from 4.6.1 to 4.8.0 URL: https://github.com/apache/nifi/pull/8567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12923 Added append avro mode to PutHDFS [nifi]
balazsgerner commented on code in PR #8544:
URL: https://github.com/apache/nifi/pull/8544#discussion_r1538866777
##
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java:
##
@@ -397,10 +432,28 @@ public void process(InputStream in) throws IOException {
fos = codec.createOutputStream(fos);
}
createdFile = actualCopyFile;
-BufferedInputStream bis = new
BufferedInputStream(in);
-StreamUtils.copy(bis, fos);
-bis = null;
-fos.flush();
+
+final String appendMode =
context.getProperty(APPEND_MODE).getValue();
+if (APPEND_RESOLUTION.equals(conflictResponse)
+&& AVRO_APPEND_MODE.equals(appendMode)
+&& destinationExists) {
+getLogger().info("Appending avro record to
existing avro file");
+try (final var reader = new
DataFileStream<>(in, new GenericDatumReader<>());
Review Comment:
Noted, thank you for the feedback. Fixed in
[04a1ed7](https://github.com/apache/nifi/pull/8544/commits/04a1ed711198fd0069e74253dacdb44b3346743c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
