Re: [PR] NIFI-12160: Kafka Connect: Check if all the necessary nars have been … [nifi]
exceptionfactory commented on code in PR #7832:
URL: https://github.com/apache/nifi/pull/7832#discussion_r1356027290
##
nifi-external/nifi-kafka-connect/nifi-kafka-connector/src/main/java/org/apache/nifi/kafka/connect/StatelessKafkaConnectorWorkingDirectoryUtil.java:
##
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.kafka.connect;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.util.Arrays;
+
+public class StatelessKafkaConnectorWorkingDirectoryUtil {
+
+public static final String NAR_UNPACKED_SUFFIX = "nar-unpacked";
+public static final String HASH_FILENAME = "nar-digest";
+private static final Logger logger =
LoggerFactory.getLogger(StatelessKafkaConnectorWorkingDirectoryUtil.class);
+
+/**
+ * Goes through the nar/extensions and extensions directories within the
working directory
+ * and deletes every directory whose name ends in "nar-unpacked" and does
not have a
+ * "nar-digest" file in it.
+ * @param workingDirectory File object pointing to the working directory.
+ */
+public static void checkWorkingDirectoryIntegrity(final File
workingDirectory) {
+purgeIncompleteUnpackedNars(new File(new File(workingDirectory,
"nar"), "extensions"));
+purgeIncompleteUnpackedNars(new File(workingDirectory, "extensions"));
+}
+
+/**
+ * Receives a directory as parameter and goes through every directory
within it that ends in
+ * "nar-unpacked". If a directory ending in "nar-unpacked" does not have a
file named
+ * "nar-digest" within it, it gets deleted with all of its contents.
+ * @param directory A File object pointing to the directory that is
supposed to contain
+ * further directories whose name ends in "nar-unpacked".
+ */
+public static void purgeIncompleteUnpackedNars(final File directory) {
+final File[] unpackedDirs = directory.listFiles(file ->
file.isDirectory() && file.getName().endsWith(NAR_UNPACKED_SUFFIX));
+if (unpackedDirs == null || unpackedDirs.length == 0) {
+logger.debug("Found no unpacked NARs in {}", directory);
+logger.debug("Directory contains: {}",
Arrays.deepToString(directory.listFiles()));
Review Comment:
The `Arrays.deepToString()` could be somewhat expensive, recommend wrapping
this log in a conditional of `isDebugEnabled()`.
##
nifi-external/nifi-kafka-connect/nifi-kafka-connector/src/main/java/org/apache/nifi/kafka/connect/StatelessKafkaConnectorWorkingDirectoryUtil.java:
##
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.kafka.connect;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.util.Arrays;
+
+public class StatelessKafkaConnectorWorkingDirectoryUtil {
+
+public static final String NAR_UNPACKED_SUFFIX = "nar-unpacked";
+public static final String HASH_FILENAME = "nar-digest";
+private static final Logger logger =
LoggerFactory.getLogger(StatelessKafkaConnectorWorkingDirectoryUtil.class);
+
+/**
+ * Goes through the nar/extensions and extensions directories within the
working directory
+ * and deletes every directory whose name ends in "nar-unpacked" and does
not have a
+ * "nar-digest" file in it.
+ * @param workingDirectory File object pointing to
[jira] [Updated] (NIFI-12215) Update Zookeeper to 3.9.1 due to CVE-2023-44981
[ https://issues.apache.org/jira/browse/NIFI-12215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12215: Summary: Update Zookeeper to 3.9.1 due to CVE-2023-44981 (was: Update Zookeeper version due to CVE-2023-44981) > Update Zookeeper to 3.9.1 due to CVE-2023-44981 > --- > > Key: NIFI-12215 > URL: https://issues.apache.org/jira/browse/NIFI-12215 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12215) Update Zookeeper version due to CVE-2023-44981
[ https://issues.apache.org/jira/browse/NIFI-12215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12215. - Resolution: Fixed > Update Zookeeper version due to CVE-2023-44981 > -- > > Key: NIFI-12215 > URL: https://issues.apache.org/jira/browse/NIFI-12215 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12215) Update Zookeeper version due to CVE-2023-44981
[ https://issues.apache.org/jira/browse/NIFI-12215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774310#comment-17774310 ] ASF subversion and git services commented on NIFI-12215: Commit 1647682709eae4fbe2f3728916c6abee5f8aeaea in nifi's branch refs/heads/support/nifi-1.x from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1647682709 ] NIFI-12215 Upgraded ZooKeeper from 3.9.0 to 3.9.1 - Resolves CVE-2023-44981 This closes #7872 Signed-off-by: David Handermann (cherry picked from commit 2bcdcab5dd6dfb03924db8b4450c3b755315dbb9) > Update Zookeeper version due to CVE-2023-44981 > -- > > Key: NIFI-12215 > URL: https://issues.apache.org/jira/browse/NIFI-12215 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12215) Update Zookeeper version due to CVE-2023-44981
[ https://issues.apache.org/jira/browse/NIFI-12215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774308#comment-17774308 ] ASF subversion and git services commented on NIFI-12215: Commit 2bcdcab5dd6dfb03924db8b4450c3b755315dbb9 in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2bcdcab5dd ] NIFI-12215 Upgraded ZooKeeper from 3.9.0 to 3.9.1 - Resolves CVE-2023-44981 This closes #7872 Signed-off-by: David Handermann > Update Zookeeper version due to CVE-2023-44981 > -- > > Key: NIFI-12215 > URL: https://issues.apache.org/jira/browse/NIFI-12215 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12215 bumped ZK version due to CVE-2023-44981 [nifi]
exceptionfactory closed pull request #7872: NIFI-12215 bumped ZK version due to CVE-2023-44981 URL: https://github.com/apache/nifi/pull/7872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-12214:
Fix Version/s: 1.24.0
(was: 1.latest)
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 2.0.0, 1.24.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-12214:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 2.0.0, 1.24.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774278#comment-17774278
]
ASF subversion and git services commented on NIFI-12214:
Commit ac435ed7a18fe1266fed97dfae2864b3ce9123ac in nifi's branch
refs/heads/support/nifi-1.x from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ac435ed7a1 ]
NIFI-12214 This closes #7869. ConsumeElasticsearch Query Builder properties do
not dependOn the Query Definition Style
Signed-off-by: Joseph Witt
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 2.0.0, 1.latest
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-12213:
Fix Version/s: 2.0.0
(was: 2.latest)
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-12214:
Fix Version/s: 2.0.0
(was: 2.latest)
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 2.0.0, 1.latest
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-12211) TinkerpopClientService is enable-able when setting the username or password in the service
[
https://issues.apache.org/jira/browse/NIFI-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt resolved NIFI-12211.
-
Fix Version/s: 2.0.0
Resolution: Fixed
> TinkerpopClientService is enable-able when setting the username or password
> in the service
> --
>
> Key: NIFI-12211
> URL: https://issues.apache.org/jira/browse/NIFI-12211
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Levi Lentz
>Assignee: Levi Lentz
>Priority: Minor
> Fix For: 2.0.0
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> In the TinkperpopClientService we are missing validators on the username and
> password as introduced by this PR:
> [https://github.com/apache/nifi/pull/7677]
> Example stacktrace when setting the username/password:
> {code:java}
> java.lang.IllegalStateException: Cannot enable Controller Service
> TestableGremlinClientService[id=gremlinService] because it is in an invalid
> state: 'user-name' validated against 'test' is invalid because 'user-name' is
> not a supported property or has no Validator associated with it
> {code}
> The fix:
> Add Validators.VALID to the username/password fields.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12211) TinkerpopClientService is enable-able when setting the username or password in the service
[
https://issues.apache.org/jira/browse/NIFI-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774274#comment-17774274
]
ASF subversion and git services commented on NIFI-12211:
Commit 1a0f509ffc4a2d0b9620d1d322807d3588deb6e4 in nifi's branch
refs/heads/main from levilentz
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1a0f509ffc ]
NIFI-12211: This closes #7868. add Validator.VALID to username/password on
TinkerpopClientService
Signed-off-by: Joseph Witt
> TinkerpopClientService is enable-able when setting the username or password
> in the service
> --
>
> Key: NIFI-12211
> URL: https://issues.apache.org/jira/browse/NIFI-12211
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Levi Lentz
>Assignee: Levi Lentz
>Priority: Minor
> Original Estimate: 24h
> Time Spent: 20m
> Remaining Estimate: 23h 40m
>
> In the TinkperpopClientService we are missing validators on the username and
> password as introduced by this PR:
> [https://github.com/apache/nifi/pull/7677]
> Example stacktrace when setting the username/password:
> {code:java}
> java.lang.IllegalStateException: Cannot enable Controller Service
> TestableGremlinClientService[id=gremlinService] because it is in an invalid
> state: 'user-name' validated against 'test' is invalid because 'user-name' is
> not a supported property or has no Validator associated with it
> {code}
> The fix:
> Add Validators.VALID to the username/password fields.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774273#comment-17774273
]
ASF subversion and git services commented on NIFI-12214:
Commit 584b3fc1652809e49f7d8f34602208ad20e6e1d0 in nifi's branch
refs/heads/main from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=584b3fc165 ]
NIFI-12214 This closes #7869. ConsumeElasticsearch Query Builder properties do
not dependOn the Query Definition Style
Signed-off-by: Joseph Witt
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774272#comment-17774272
]
ASF subversion and git services commented on NIFI-12213:
Commit f2927525f30934c9566b0d0b400b268cac0e9297 in nifi's branch
refs/heads/main from Michael W Moser
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f2927525f3 ]
NIFI-12213 This closes #7871. add nifi-utils dependency to NARs that do not use
nifi-standard-services-api-nar as a parent
Signed-off-by: Joseph Witt
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12213 add nifi-utils dependency to NARs that do not use nifi-sta… [nifi]
asfgit closed pull request #7871: NIFI-12213 add nifi-utils dependency to NARs that do not use nifi-sta… URL: https://github.com/apache/nifi/pull/7871 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12211: add Validator.VALID to username/password on TinkerpopClientService [nifi]
asfgit closed pull request #7868: NIFI-12211: add Validator.VALID to username/password on TinkerpopClientService URL: https://github.com/apache/nifi/pull/7868 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12214 ConsumeElasticsearch Query Builder properties do not dependOn the Query Definition Style [nifi]
asfgit closed pull request #7869: NIFI-12214 ConsumeElasticsearch Query Builder properties do not dependOn the Query Definition Style URL: https://github.com/apache/nifi/pull/7869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12215 bumped ZK version due to CVE-2023-44981 [nifi]
joewitt opened a new pull request, #7872: URL: https://github.com/apache/nifi/pull/7872 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12215) Update Zookeeper version due to CVE-2023-44981
Joe Witt created NIFI-12215: --- Summary: Update Zookeeper version due to CVE-2023-44981 Key: NIFI-12215 URL: https://issues.apache.org/jira/browse/NIFI-12215 Project: Apache NiFi Issue Type: Task Components: Core Framework Reporter: Joe Witt Assignee: Joe Witt Fix For: 2.0.0, 1.24.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12174) Upgrade AWS Lambda Processors to use AWS 2.x libraries
[ https://issues.apache.org/jira/browse/NIFI-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774246#comment-17774246 ] David Handermann commented on NIFI-12174: - Thanks for working on this [~jgresock]. With other changes, this was not a simple backport for the support branch, but it could be done under a separate Jira issue given that the SDK library is also available. > Upgrade AWS Lambda Processors to use AWS 2.x libraries > -- > > Key: NIFI-12174 > URL: https://issues.apache.org/jira/browse/NIFI-12174 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12174) Upgrade AWS Lambda Processors to use AWS 2.x libraries
[ https://issues.apache.org/jira/browse/NIFI-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12174. - Fix Version/s: 2.0.0 (was: 2.latest) Resolution: Fixed > Upgrade AWS Lambda Processors to use AWS 2.x libraries > -- > > Key: NIFI-12174 > URL: https://issues.apache.org/jira/browse/NIFI-12174 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12174) Upgrade AWS Lambda Processors to use AWS 2.x libraries
[ https://issues.apache.org/jira/browse/NIFI-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774244#comment-17774244 ] ASF subversion and git services commented on NIFI-12174: Commit 9627c4c0e85403c8da18c1a9a6910aa71f656bca in nifi's branch refs/heads/main from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9627c4c0e8 ] NIFI-12174 Upgraded AWS PutLambda Processor to SDK 2 This closes #7851 Signed-off-by: David Handermann > Upgrade AWS Lambda Processors to use AWS 2.x libraries > -- > > Key: NIFI-12174 > URL: https://issues.apache.org/jira/browse/NIFI-12174 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Minor > Fix For: 2.latest > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12174: Upgrading AWS Lambda processor to SDK 2.x [nifi]
exceptionfactory closed pull request #7851: NIFI-12174: Upgrading AWS Lambda processor to SDK 2.x URL: https://github.com/apache/nifi/pull/7851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774243#comment-17774243
]
David Handermann commented on NIFI-12213:
-
Thanks [~mosermw], the pull request looks good and I will plan on merging
pending successful builds.
The {{nifi-standard-services-api-nar}} bundles {{{}nifi-utils{}}}, so most
modules had it in the class path at runtime, but a few don't as you found.
Those two modules were an oversight on my part with the restructuring. Some
modules had direct dependencies on nifi-utils, others had transitive
dependencies. The transitive dependencies worked when nifi-utils had the
default {{compile}} scope, but with the addition of the {{{}nifi-bom{}}}, it is
now necessary to declare {{nifi-utils}} explicitly. This should be better going
forward, as it surfaces direct dependencies versus relying on implicit
relationships.
Thanks again for the quick turnaround!
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12165) JoltTransformJSON Processor - Vanishing properties
[ https://issues.apache.org/jira/browse/NIFI-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774240#comment-17774240 ] Michael W Moser commented on NIFI-12165: Confirmed. The properties "Custom Transformation Class Name" and "Custom Module Directory" should depend on the "Jolt Transformation DSL = Custom". However, the dependency is incorrectly set to depend on "Jolt Specification" property. As an experiment, if you set "Jolt Specification = jolt-transform-custom", you will see the two custom properties come back. > JoltTransformJSON Processor - Vanishing properties > -- > > Key: NIFI-12165 > URL: https://issues.apache.org/jira/browse/NIFI-12165 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.21.0 > Environment: Docker >Reporter: Benji Benning >Priority: Minor > Attachments: JoltTransformJSON.mp4 > > > When selecting any Property to edit (then either accepting or canceling the > change) the "Custom Transformation Class Name" and the "Custom Module > Directory" properties vanish. > Note that the change does seem to get applied. > See attached video -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774235#comment-17774235
]
Michael W Moser commented on NIFI-12213:
I didn't find any other places where nifi-utils needed an explicit dependency.
I did find that nifi-twitter-processors and nifi-networking-processors have an
explicit dependency on nifi-utils that they probably don't need, but leaving
them alone doesn't harm anything (nifi-utils does not get included in their
nar).
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12213:
---
Status: Patch Available (was: Open)
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12213 add nifi-utils dependency to NARs that do not use nifi-sta… [nifi]
mosermw opened a new pull request, #7871: URL: https://github.com/apache/nifi/pull/7871 NIFI-12213 add nifi-utils dependency to NARs that do not use nifi-standard-services-api-nar as a parent # Summary [NIFI-12213](https://issues.apache.org/jira/browse/NIFI-12213) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [n/a] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [n/a] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser reassigned NIFI-12213:
--
Assignee: Michael W Moser
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12206) Refactor Flow Configuration History from H2 to JetBrains Xodus
[ https://issues.apache.org/jira/browse/NIFI-12206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12206: Status: Patch Available (was: In Progress) > Refactor Flow Configuration History from H2 to JetBrains Xodus > -- > > Key: NIFI-12206 > URL: https://issues.apache.org/jira/browse/NIFI-12206 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > The [H2 Database Engine|https://h2database.com/html/main.html] has provided > local persistent storage for several types of information across Apache NiFi > versions. With multiple refactoring efforts over several versions, H2 no > longer stores user session information or identity provider group > information, leaving the Flow Configuration History as the only remaining > reference to H2. > H2 version 2.1 introduced incompatible changes in the binary storage format > from H2 version 1.4, and H2 version 2.2 was also unable to read files from > earlier H2 versions. These binary changes required custom migration modules > and shaded distribution of H2 libraries to support upgrading between Apache > NiFi versions. > With the scope of H2 usage narrowed to Flow Configuration History in Apache > NiFi 1.23.0 and following, the storage strategy should be changed. Apache > Derby, SQLite, and HSQLDB are other potential options supporting file-based > relational storage, but maintenance level and platform-specific limitations > present concerns with these libraries. > The [JetBrains Xodus|https://github.com/JetBrains/xodus] library provides > persistent and scalable storage that avoids several issues present in other > alternatives. The framework is licensed under Apache Software License Version > 2.0 and has a narrow set of dependencies aside from the Kotlin standard > libraries. Xodus is now in version 2.0 and has maintained format > compatibility when upgrading between major versions. > Based on the H2 database migration modules on the Apache NiFi support branch, > a subsequent issue can implement automated migration from H2 to Xodus, > providing an upgrade path from NiFi 1 to 2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12206 Refactor Flow History using JetBrains Xodus [nifi]
exceptionfactory opened a new pull request, #7870: URL: https://github.com/apache/nifi/pull/7870 # Summary [NIFI-12206](https://issues.apache.org/jira/browse/NIFI-12206) Refactors the Flow Configuration History auditing to use the JetBrains [Xodus](https://github.com/JetBrains/xodus) library for persistent storage in place of the [H2 Database Engine](https://h2database.com/html/main.html). These changes are targeted for the main branch, and follow-on work will handle automated migration for the support branch. The implementation removes the existing `StandardAuditService` and adds an `EntityStoreAuditService` with transactional operations using an Xodus [Entity Store](https://github.com/JetBrains/xodus/wiki/Entity-Stores). Changes include removing the `nifi.h2.url.append` property from `nifi.properties`. The new `EntityStoreAuditService` includes exception handling when loading to avoid application startup failures for corrupted files. The exception handling approach renames files in the database directory with a backup timestamp extension, and then proceeds to create a new instance. This strategy allows the potential for manual recovery, while enabling the application to start with a new persistent entity store. The new implementation supports existing Flow Configuration History queries with results filtering and paging. Queries against a test environment containing over one million entries remained responsive for both filtered and unfiltered results. JetBrains Xodus is licensed under Apache Software License Version 2.0 and does not contain additional notices. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [X] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [X] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [X] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-8135 allow CHOICE data types in conversion of Records to Java Maps [nifi]
ChrisSamo632 commented on code in PR #7746:
URL: https://github.com/apache/nifi/pull/7746#discussion_r1355749268
##
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java:
##
@@ -854,79 +855,82 @@ public static Map toMap(final Object
value, final String fieldNa
*/
@SuppressWarnings({"unchecked", "rawtypes"})
public static Object convertRecordFieldtoObject(final Object value, final
DataType dataType) {
-
if (value == null) {
return null;
}
+DataType chosenDataType;
+if (dataType instanceof ChoiceDataType) {
+final DataType chosen = chooseDataType(value, (ChoiceDataType)
dataType);
+chosenDataType = chosen != null ? chosen : dataType;
+} else {
+chosenDataType = dataType;
+}
+
if (value instanceof Record) {
-Record record = (Record) value;
-RecordSchema recordSchema = record.getSchema();
+final Record record = (Record) value;
+final RecordSchema recordSchema = record.getSchema();
if (recordSchema == null) {
throw new IllegalTypeConversionException("Cannot convert value
of type Record to Map because Record does not have an associated Schema");
}
-final Map recordMap = new LinkedHashMap<>();
-for (RecordField field : recordSchema.getFields()) {
-final DataType fieldDataType = field.getDataType();
+final Map recordMap = new
LinkedHashMap<>(record.getRawFieldNames().size(), 1);
+for (final RecordField field : recordSchema.getFields()) {
+if (field.getDataType() instanceof ChoiceDataType) {
+final DataType chosen = chooseDataType(value,
(ChoiceDataType) field.getDataType());
+chosenDataType = chosen != null ? chosen :
field.getDataType();
+} else {
+chosenDataType = field.getDataType();
+}
final String fieldName = field.getFieldName();
-Object fieldValue = record.getValue(fieldName);
+final Object fieldValue = record.getValue(fieldName);
if (fieldValue == null) {
recordMap.put(fieldName, null);
-} else if (isScalarValue(fieldDataType, fieldValue)) {
+} else if (isScalarValue(chosenDataType, fieldValue)) {
recordMap.put(fieldName, fieldValue);
-} else if (fieldDataType instanceof RecordDataType) {
+} else if (chosenDataType instanceof RecordDataType) {
Record nestedRecord = (Record) fieldValue;
Review Comment:
Resolved, the `chooseDataType` call needed to use the `fieldValue`, not the
`value` (of the Record passed to the method)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12172 - Update README.md adding OpenID server for Nifi Registry [nifi]
marcelo225 commented on PR #7839: URL: https://github.com/apache/nifi/pull/7839#issuecomment-1758484763 @exceptionfactory thank you for helping me. I've committed the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774224#comment-17774224
]
David Handermann commented on NIFI-12213:
-
Thanks for tracking down this issue [~mosermw]!
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-12214:
-
Status: Patch Available (was: In Progress)
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12214 ConsumeElasticsearch Query Builder properties do not dependOn the Query Definition Style [nifi]
ChrisSamo632 opened a new pull request, #7869: URL: https://github.com/apache/nifi/pull/7869 # Summary [NIFI-12214](https://issues.apache.org/jira/browse/NIFI-12214) ConsumeElasticsearch Query Builder properties do not dependOn the Query Definition Style # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 21 ### Licensing - ~[ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)~ - ~[ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files~ ### Documentation - ~[ ] Documentation formatting appears as expected in rendered files~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson reassigned NIFI-12214:
Assignee: Chris Sampson
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 1.latest, 2.latest
>
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12214) ConsumeElasticsearch doc generation broken
[
https://issues.apache.org/jira/browse/NIFI-12214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774214#comment-17774214
]
ASF subversion and git services commented on NIFI-12214:
Commit 2282a96c615ca8191b17bdd7d4fbdc1a4545dea2 in nifi's branch
refs/heads/NIFI-12214 from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2282a96c61 ]
NIFI-12214 ConsumeElasticsearch Query Builder properties do not dependOn the
Query Definition Style
> ConsumeElasticsearch doc generation broken
> --
>
> Key: NIFI-12214
> URL: https://issues.apache.org/jira/browse/NIFI-12214
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.24.0
>Reporter: Chris Sampson
>Priority: Minor
> Fix For: 1.latest, 2.latest
>
>
> The NiFi DocGenerator is throwing a WARNing during startup when processing
> the {{ConsumeElasticsearch}} properties.
> {code:java}
> java.io.IOException: Unable to create XMLOutputStream
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
> at
> org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
> at
> org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
> at
> org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
> at org.apache.nifi.NiFi.(NiFi.java:172)
> at org.apache.nifi.NiFi.(NiFi.java:83)
> at org.apache.nifi.NiFi.main(NiFi.java:332)
> Caused by: javax.xml.stream.XMLStreamException: No property was found
> matching the name 'el-rest-query-definition-style'
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
> at
> org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
> ... 7 common frames omitted
> {code}
> This is due to the Query Builder properties inherrited from
> {{ElasticsearchRestProcessor}} that {{dependOn}} the
> {{QUERY_DEFINITION_STYLE}} property, which is omitted from
> {{ConsumeElasticsearch}} because it doesn't make sense to use (one must build
> the Elasticsearch {{query}} from the Query Builder properties rather than
> defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12214) ConsumeElasticsearch doc generation broken
Chris Sampson created NIFI-12214:
Summary: ConsumeElasticsearch doc generation broken
Key: NIFI-12214
URL: https://issues.apache.org/jira/browse/NIFI-12214
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.0.0, 1.24.0
Reporter: Chris Sampson
Fix For: 1.latest, 2.latest
The NiFi DocGenerator is throwing a WARNing during startup when processing the
{{ConsumeElasticsearch}} properties.
{code:java}
java.io.IOException: Unable to create XMLOutputStream
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:102)
at
org.apache.nifi.documentation.DocGenerator.document(DocGenerator.java:142)
at
org.apache.nifi.documentation.DocGenerator.documentConfigurableComponent(DocGenerator.java:107)
at
org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:64)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:763)
at org.apache.nifi.NiFi.(NiFi.java:172)
at org.apache.nifi.NiFi.(NiFi.java:83)
at org.apache.nifi.NiFi.main(NiFi.java:332)
Caused by: javax.xml.stream.XMLStreamException: No property was found matching
the name 'el-rest-query-definition-style'
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeProperties(HtmlDocumentationWriter.java:743)
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.writeBody(HtmlDocumentationWriter.java:166)
at
org.apache.nifi.documentation.html.HtmlDocumentationWriter.write(HtmlDocumentationWriter.java:98)
... 7 common frames omitted
{code}
This is due to the Query Builder properties inherrited from
{{ElasticsearchRestProcessor}} that {{dependOn}} the {{QUERY_DEFINITION_STYLE}}
property, which is omitted from {{ConsumeElasticsearch}} because it doesn't
make sense to use (one must build the Elasticsearch {{query}} from the Query
Builder properties rather than defining the whole {{query}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-4612: Allow AvroSchemaRegistry to disable name validation [nifi]
AdityaSurve96 commented on PR #2275: URL: https://github.com/apache/nifi/pull/2275#issuecomment-1758426928 Hi @mattyb149 , other apps dont work with invalid schema . What I mean in until now we did not have any case where avro schema field name had dash ( hiphen) in it. But now in some use cases we do have field names with dash ( hiphen ) which does not get compiled by Avro. So can you please help me to understand where and how can I implement / incorporate the given solution in this ticket to disable name validation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774205#comment-17774205
]
Michael W Moser edited comment on NIFI-12213 at 10/11/23 7:19 PM:
--
I added nifi-utils as a compile dependency in the nifi-ldap-iaa-providers-nar
pom.xml and this fixes it.
I wonder if any other nar is affected?
[edit] looks like nifi-kerberos-iaa-providers-nar needs this too
was (Author: mosermw):
I added nifi-utils as a compile dependency in the nifi-ldap-iaa-providers-nar
pom.xml and this fixes it.
I wonder if any other nar is affected?
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774205#comment-17774205
]
Michael W Moser commented on NIFI-12213:
I added nifi-utils as a compile dependency in the nifi-ldap-iaa-providers-nar
pom.xml and this fixes it.
I wonder if any other nar is affected?
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
Michael W Moser created NIFI-12213:
--
Summary: Since nifi-bom was introduced, using
LdapUserGroupProvider causes NiFi to fail to start
Key: NIFI-12213
URL: https://issues.apache.org/jira/browse/NIFI-12213
Project: Apache NiFi
Issue Type: Bug
Reporter: Michael W Moser
Fix For: 2.latest
I built the main branch and tried to use the LdapUserGroupProvider in
authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
on nifi-standard-services-api-nar.
{noformat}
Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
at
org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
at
org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at
org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
at
org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
at
org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
at
org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
at
org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
... 106 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
... 116 common frames omitted
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12211: add Validator.VALID to username/password on TinkerpopClientService [nifi]
ChrisSamo632 commented on code in PR #7868: URL: https://github.com/apache/nifi/pull/7868#discussion_r1355627921 ## nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java: ## @@ -167,6 +167,7 @@ public class TinkerpopClientService extends AbstractControllerService implements " Note: when using a remote.yaml file, this username value (if set) will overload any " + "username set in the YAML file.") .required(false) +.addValidator(Validator.VALID) Review Comment: Seems like `StandardValidators.ATTRIBUTE_EXPRESSION_LANGUAGE_VALIDATOR` or `StandardValidators.NON_EMPTY_VALIDATOR` would be better for these properties - this allows them to remain optional, but prevents empty values being set, which is probably not valid -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12203) JSON Sync does not delete controller level items
[ https://issues.apache.org/jira/browse/NIFI-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774199#comment-17774199 ] ASF subversion and git services commented on NIFI-12203: Commit 3f13604c365ac9205b9a8dc2782bf13a08b76b26 in nifi's branch refs/heads/main from Simon Bence [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3f13604c36 ] NIFI-11772 Removed flow.xml.gz support - Created NIFI-12203 to evaluate issues with flow comparison surfaced in JoinClusterWithDifferentFlow This closes #7661 Signed-off-by: David Handermann > JSON Sync does not delete controller level items > > > Key: NIFI-12203 > URL: https://issues.apache.org/jira/browse/NIFI-12203 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 2.0.0 >Reporter: Simon Bence >Priority: Blocker > > JSON based flow synchronization looks to behave differently than XML in > regards of controllel level components: while XML cleans up "removed" > components from the current flow, the JSON keeps them. This happens in case > of > [JoinClusterWithDifferentFlow|https://github.com/apache/nifi/blob/main/nifi-system-tests/nifi-system-test-suite/src/test/java/org/apache/nifi/tests/system/clustering/JoinClusterWithDifferentFlow.java#L240] > where components within the root group are "replaced" as the tests expects > but the controller level Controller Service is not. > Running the test using JSON synchronizer results two Controller Services in > the memory representation of the flow definition on node 2: both the services > from flow1 and flow2 are part of the synced flow. > After some testing, it looks like > [this|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/serialization/VersionedFlowSynchronizer.java#L900] > method and other methods in the "inherit*" method family will not clean up > components are not present in the proposed flow. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11772) Remove flow.xml support from NiFi 2.0
[ https://issues.apache.org/jira/browse/NIFI-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774198#comment-17774198 ] ASF subversion and git services commented on NIFI-11772: Commit 3f13604c365ac9205b9a8dc2782bf13a08b76b26 in nifi's branch refs/heads/main from Simon Bence [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3f13604c36 ] NIFI-11772 Removed flow.xml.gz support - Created NIFI-12203 to evaluate issues with flow comparison surfaced in JoinClusterWithDifferentFlow This closes #7661 Signed-off-by: David Handermann > Remove flow.xml support from NiFi 2.0 > - > > Key: NIFI-11772 > URL: https://issues.apache.org/jira/browse/NIFI-11772 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Time Spent: 4h 10m > Remaining Estimate: 0h > > For a while, the JSON based flow persistance is the preferred approach and as > part of the effort to clean up deprecated code from NiFi, the flow.xml > support needs to be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11772) Remove flow.xml support from NiFi 2.0
[ https://issues.apache.org/jira/browse/NIFI-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11772. - Fix Version/s: 2.0.0 Resolution: Fixed > Remove flow.xml support from NiFi 2.0 > - > > Key: NIFI-11772 > URL: https://issues.apache.org/jira/browse/NIFI-11772 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 2.0.0 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > For a while, the JSON based flow persistance is the preferred approach and as > part of the effort to clean up deprecated code from NiFi, the flow.xml > support needs to be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-11772 Removing flow.xml support [nifi]
exceptionfactory closed pull request #7661: NIFI-11772 Removing flow.xml support URL: https://github.com/apache/nifi/pull/7661 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12212) Upgrade AWS DynamoDB Processors to use AWS 2.x libraries
Joe Gresock created NIFI-12212: -- Summary: Upgrade AWS DynamoDB Processors to use AWS 2.x libraries Key: NIFI-12212 URL: https://issues.apache.org/jira/browse/NIFI-12212 Project: Apache NiFi Issue Type: Improvement Reporter: Joe Gresock Assignee: Joe Gresock Fix For: 2.latest -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12207) Upgrade Netty to 4.1.100
[ https://issues.apache.org/jira/browse/NIFI-12207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12207: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Netty to 4.1.100 > > > Key: NIFI-12207 > URL: https://issues.apache.org/jira/browse/NIFI-12207 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: dependency-upgrade > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to 4.1.100 to mitigate > [CVE-2023-44487|https://www.cve.org/CVERecord?id=CVE-2023-44487], related to > potential Denial of Service for server components supporting HTTP/2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12207) Upgrade Netty to 4.1.100
[ https://issues.apache.org/jira/browse/NIFI-12207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774168#comment-17774168 ] ASF subversion and git services commented on NIFI-12207: Commit 9caca92a83647700c1da83c071dd872cae76cd1d in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9caca92a83 ] NIFI-12207 Upgraded Netty from 4.1.99 to 4.1.100 Signed-off-by: Pierre Villard This closes #7865. > Upgrade Netty to 4.1.100 > > > Key: NIFI-12207 > URL: https://issues.apache.org/jira/browse/NIFI-12207 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: dependency-upgrade > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to 4.1.100 to mitigate > [CVE-2023-44487|https://www.cve.org/CVERecord?id=CVE-2023-44487], related to > potential Denial of Service for server components supporting HTTP/2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12207) Upgrade Netty to 4.1.100
[ https://issues.apache.org/jira/browse/NIFI-12207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774167#comment-17774167 ] ASF subversion and git services commented on NIFI-12207: Commit 1b57980067416187dd068adc35e9577bca3dbb8b in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1b57980067 ] NIFI-12207 Upgraded Netty from 4.1.99 to 4.1.100 Signed-off-by: Pierre Villard This closes #7865. > Upgrade Netty to 4.1.100 > > > Key: NIFI-12207 > URL: https://issues.apache.org/jira/browse/NIFI-12207 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: dependency-upgrade > Fix For: 2.0.0, 1.24.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to 4.1.100 to mitigate > [CVE-2023-44487|https://www.cve.org/CVERecord?id=CVE-2023-44487], related to > potential Denial of Service for server components supporting HTTP/2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12207 Upgrade Netty from 4.1.99 to 4.1.100 [nifi]
asfgit closed pull request #7865: NIFI-12207 Upgrade Netty from 4.1.99 to 4.1.100 URL: https://github.com/apache/nifi/pull/7865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12208) Upgrade Jetty to 10.0.17
[ https://issues.apache.org/jira/browse/NIFI-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12208: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Jetty to 10.0.17 > > > Key: NIFI-12208 > URL: https://issues.apache.org/jira/browse/NIFI-12208 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Jetty dependencies on the main branch should be upgraded to > [10.0.17|https://github.com/eclipse/jetty.project/releases/tag/jetty-10.0.17] > to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12208) Upgrade Jetty to 10.0.17
[ https://issues.apache.org/jira/browse/NIFI-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774165#comment-17774165 ] ASF subversion and git services commented on NIFI-12208: Commit b236200a84991559823954165484df9de2bf9d79 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b236200a84 ] NIFI-12208 Upgraded Jetty from 10.0.16 to 10.0.17 Signed-off-by: Pierre Villard This closes #7866. > Upgrade Jetty to 10.0.17 > > > Key: NIFI-12208 > URL: https://issues.apache.org/jira/browse/NIFI-12208 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Jetty dependencies on the main branch should be upgraded to > [10.0.17|https://github.com/eclipse/jetty.project/releases/tag/jetty-10.0.17] > to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12208 Upgrade Jetty from 10.0.16 to 10.0.17 [nifi]
asfgit closed pull request #7866: NIFI-12208 Upgrade Jetty from 10.0.16 to 10.0.17 URL: https://github.com/apache/nifi/pull/7866 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12209) Upgrade Jetty to 9.4.53 on Support Branch
[ https://issues.apache.org/jira/browse/NIFI-12209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12209: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Jetty to 9.4.53 on Support Branch > - > > Key: NIFI-12209 > URL: https://issues.apache.org/jira/browse/NIFI-12209 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: dependency-upgrade > Fix For: 1.24.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Jetty dependencies on the support branch should be upgraded to > [9.4.53|https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.53.v20231009] > to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12209 Upgrade Jetty from 9.4.52 to 9.4.53 [nifi]
pvillard31 closed pull request #7867: NIFI-12209 Upgrade Jetty from 9.4.52 to 9.4.53 URL: https://github.com/apache/nifi/pull/7867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12209 Upgrade Jetty from 9.4.52 to 9.4.53 [nifi]
pvillard31 commented on PR #7867: URL: https://github.com/apache/nifi/pull/7867#issuecomment-1758187002 Merged to 1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12209) Upgrade Jetty to 9.4.53 on Support Branch
[ https://issues.apache.org/jira/browse/NIFI-12209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774163#comment-17774163 ] ASF subversion and git services commented on NIFI-12209: Commit e7888d177ccbc06ae796778399730113d263fe1f in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e7888d177c ] NIFI-12209 Upgraded Jetty from 9.4.52 to 9.4.53 Signed-off-by: Pierre Villard This closes #7867. > Upgrade Jetty to 9.4.53 on Support Branch > - > > Key: NIFI-12209 > URL: https://issues.apache.org/jira/browse/NIFI-12209 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: dependency-upgrade > Fix For: 1.24.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Jetty dependencies on the support branch should be upgraded to > [9.4.53|https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.53.v20231009] > to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12211: add Validator.VALID to username/password on TinkerpopClientService [nifi]
levilentz opened a new pull request, #7868: URL: https://github.com/apache/nifi/pull/7868 # Summary This fixes a bug that was introduced in https://github.com/apache/nifi/pull/7677 [NIFI-12211](https://issues.apache.org/jira/browse/NIFI-12211) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [X] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [X] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [X] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12211) TinkerpopClientService is enable-able when setting the username or password in the service
Levi Lentz created NIFI-12211:
-
Summary: TinkerpopClientService is enable-able when setting the
username or password in the service
Key: NIFI-12211
URL: https://issues.apache.org/jira/browse/NIFI-12211
Project: Apache NiFi
Issue Type: Bug
Reporter: Levi Lentz
Assignee: Levi Lentz
In the TinkperpopClientService we are missing validators on the username and
password as introduced by this PR:
[https://github.com/apache/nifi/pull/7677]
Example stacktrace when setting the username/password:
{code:java}
java.lang.IllegalStateException: Cannot enable Controller Service
TestableGremlinClientService[id=gremlinService] because it is in an invalid
state: 'user-name' validated against 'test' is invalid because 'user-name' is
not a supported property or has no Validator associated with it
{code}
The fix:
Add Validators.VALID to the username/password fields.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] MINIFICPP-2239 Add bandwidth limit to InvokeHTTP processor [nifi-minifi-cpp]
szaszm commented on code in PR #1674: URL: https://github.com/apache/nifi-minifi-cpp/pull/1674#discussion_r1355153756 ## PROCESSORS.md: ## @@ -1320,6 +1320,8 @@ In the list below, the names of required properties appear in bold. Any other pr | Always Output Response | false| truefalse | Will force a response FlowFile to be generated and routed to the 'Response' relationship regardless of what the server status code received is | | Penalize on "No Retry" | false| truefalse | Enabling this property will penalize FlowFiles that are routed to the "No Retry" relationship. | | **Invalid HTTP Header Field Handling Strategy** | transform| failtransformdrop | Indicates what should happen when an attribute's name is not a valid HTTP header field name. Options: transform - invalid characters are replaced, fail - flow file is transferred to failure, drop - drops invalid attributes from HTTP message | +| Upload Speed Limit | | | Maximum data per second to send (e.g. '500 KB/s'). Leave this empty if you want no limit. | +| Download Speed Limit| | | Maximum data per second to receive (e.g. '500 KB/s'). Leave this empty if you want no limit. | Review Comment: ```suggestion | Upload Speed Limit | | | Maximum upload speed, e.g. '500 KB/s'. Leave this empty if you want no limit. | | Download Speed Limit| | | Maximum download speed,e.g. '500 KB/s'. Leave this empty if you want no limit. | ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2239 Add bandwidth limit to InvokeHTTP processor [nifi-minifi-cpp]
lordgamez commented on code in PR #1674:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1674#discussion_r1355279672
##
libminifi/include/core/TypedValues.h:
##
@@ -149,10 +144,36 @@ class DataSizeValue : public TransformableValue, public
state::response::UInt64V
}
};
-} // namespace core
-} // namespace minifi
-} // namespace nifi
-} // namespace apache
-} // namespace org
+class DataTransferSpeedValue : public TransformableValue, public
state::response::UInt64Value {
+ public:
+ MINIFIAPI static const std::type_index type_id;
+
+ explicit DataTransferSpeedValue(const std::string _speed_string)
+ : state::response::UInt64Value(0) {
+
DataSizeValue::StringToInt(removePerSecSuffix(transfer_speed_string),
value);
+string_value = transfer_speed_string;
+ }
+
+ explicit DataTransferSpeedValue(uint64_t value)
+ : state::response::UInt64Value(value) {
+ }
+
+ DataTransferSpeedValue()
+ : state::response::UInt64Value(0) {
+ }
+
+ static std::string removePerSecSuffix(const std::string ) {
+auto lower_case_input =
utils::StringUtils::trim(utils::StringUtils::toLower(input));
+if (lower_case_input.ends_with("/s")) {
+ return lower_case_input.substr(0, lower_case_input.size() - 2);
+}
+throw utils::internal::ParseException("Couldn't parse
DataTransferSpeedValue, no valid suffix (/s, /sec, ps) found!");
Review Comment:
Updated in 41364556e36680ca7f8d564ffa08fe110188a34c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2239 Add bandwidth limit to InvokeHTTP processor [nifi-minifi-cpp]
lordgamez commented on code in PR #1674: URL: https://github.com/apache/nifi-minifi-cpp/pull/1674#discussion_r1355279283 ## docker/test/integration/features/MiNiFi_integration_test_driver.py: ## @@ -244,11 +246,21 @@ def check_for_num_files_generated(self, num_flowfiles, timeout_seconds): output_validator.set_output_dir(self.file_system_observer.get_output_dir()) self.__check_output(timeout_seconds, output_validator, max(1, num_flowfiles)) -def check_for_num_file_range_generated(self, min_files, max_files, wait_time_in_seconds): +def check_for_num_file_range_generated_after_timeout(self, min_files: int, max_files: int, wait_time_in_seconds: int): output_validator = NumFileRangeValidator(min_files, max_files) output_validator.set_output_dir(self.file_system_observer.get_output_dir()) self.__check_output_after_time_period(wait_time_in_seconds, output_validator) +def check_for_num_file_range_generated(self, min_files: int, max_files: int, wait_time_in_seconds: int): +output_validator = NumFileRangeValidator(min_files, max_files) + output_validator.set_output_dir(self.file_system_observer.get_output_dir()) +self.__check_output_over_time_period(wait_time_in_seconds, output_validator) Review Comment: Updated in 41364556e36680ca7f8d564ffa08fe110188a34c ## extensions/http-curl/processors/InvokeHTTP.cpp: ## @@ -115,6 +115,8 @@ void InvokeHTTP::setupMembersFromProperties(const core::ProcessContext& context) use_chunked_encoding_ = (context.getProperty(UseChunkedEncoding) | utils::andThen(::StringUtils::toBool)).value_or(false); send_date_header_ = context.getProperty(DateHeader).value_or(true); + context.getProperty(UploadSpeedLimit, maximum_upload_speed_); + context.getProperty(DownloadSpeedLimit, maximum_download_speed_); Review Comment: Good catch, updated in 41364556e36680ca7f8d564ffa08fe110188a34c -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2239 Add bandwidth limit to InvokeHTTP processor [nifi-minifi-cpp]
fgerlits commented on code in PR #1674:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1674#discussion_r1355024576
##
docker/test/integration/features/MiNiFi_integration_test_driver.py:
##
@@ -244,11 +246,21 @@ def check_for_num_files_generated(self, num_flowfiles,
timeout_seconds):
output_validator.set_output_dir(self.file_system_observer.get_output_dir())
self.__check_output(timeout_seconds, output_validator, max(1,
num_flowfiles))
-def check_for_num_file_range_generated(self, min_files, max_files,
wait_time_in_seconds):
+def check_for_num_file_range_generated_after_timeout(self, min_files: int,
max_files: int, wait_time_in_seconds: int):
output_validator = NumFileRangeValidator(min_files, max_files)
output_validator.set_output_dir(self.file_system_observer.get_output_dir())
self.__check_output_after_time_period(wait_time_in_seconds,
output_validator)
+def check_for_num_file_range_generated(self, min_files: int, max_files:
int, wait_time_in_seconds: int):
+output_validator = NumFileRangeValidator(min_files, max_files)
+
output_validator.set_output_dir(self.file_system_observer.get_output_dir())
+self.__check_output_over_time_period(wait_time_in_seconds,
output_validator)
Review Comment:
the difference between these two could be made clearer:
```suggestion
def check_for_num_file_range_generated_after_wait(self, min_files: int,
max_files: int, wait_time_in_seconds: int):
output_validator = NumFileRangeValidator(min_files, max_files)
output_validator.set_output_dir(self.file_system_observer.get_output_dir())
self.__check_output_after_time_period(wait_time_in_seconds,
output_validator)
def check_for_num_file_range_generated_with_timeout(self, min_files:
int, max_files: int, timeout_in_seconds: int):
output_validator = NumFileRangeValidator(min_files, max_files)
output_validator.set_output_dir(self.file_system_observer.get_output_dir())
self.__check_output_over_time_period(timeout_in_seconds,
output_validator)
```
##
extensions/http-curl/processors/InvokeHTTP.cpp:
##
@@ -115,6 +115,8 @@ void InvokeHTTP::setupMembersFromProperties(const
core::ProcessContext& context)
use_chunked_encoding_ = (context.getProperty(UseChunkedEncoding) |
utils::andThen(::StringUtils::toBool)).value_or(false);
send_date_header_ = context.getProperty(DateHeader).value_or(true);
+ context.getProperty(UploadSpeedLimit, maximum_upload_speed_);
+ context.getProperty(DownloadSpeedLimit, maximum_download_speed_);
Review Comment:
These will parse the the properties as numbers, ignoring the suffix, so
`Download Speed Limit = 10 MB/s` will result in `maximum_download_speed_ = 10`
instead of `maximum_download_speed_ = 1000`.
If you changed the type of these two fields to
`core::DataTransferSpeedValue`, then it would work. In that case, we would
need to call `getValue()` when passing the values to curl.
##
libminifi/include/core/TypedValues.h:
##
@@ -149,10 +144,36 @@ class DataSizeValue : public TransformableValue, public
state::response::UInt64V
}
};
-} // namespace core
-} // namespace minifi
-} // namespace nifi
-} // namespace apache
-} // namespace org
+class DataTransferSpeedValue : public TransformableValue, public
state::response::UInt64Value {
+ public:
+ MINIFIAPI static const std::type_index type_id;
+
+ explicit DataTransferSpeedValue(const std::string _speed_string)
+ : state::response::UInt64Value(0) {
+
DataSizeValue::StringToInt(removePerSecSuffix(transfer_speed_string),
value);
+string_value = transfer_speed_string;
+ }
+
+ explicit DataTransferSpeedValue(uint64_t value)
+ : state::response::UInt64Value(value) {
+ }
+
+ DataTransferSpeedValue()
+ : state::response::UInt64Value(0) {
+ }
+
+ static std::string removePerSecSuffix(const std::string ) {
+auto lower_case_input =
utils::StringUtils::trim(utils::StringUtils::toLower(input));
+if (lower_case_input.ends_with("/s")) {
+ return lower_case_input.substr(0, lower_case_input.size() - 2);
+}
+throw utils::internal::ParseException("Couldn't parse
DataTransferSpeedValue, no valid suffix (/s, /sec, ps) found!");
Review Comment:
this error message is out of date
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-12210) Implement recovery mode for WriteAheadFlowFileRepository
[ https://issues.apache.org/jira/browse/NIFI-12210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynaldo Rea reassigned NIFI-12210: --- Assignee: Reynaldo Rea > Implement recovery mode for WriteAheadFlowFileRepository > > > Key: NIFI-12210 > URL: https://issues.apache.org/jira/browse/NIFI-12210 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Reynaldo Rea >Assignee: Reynaldo Rea >Priority: Major > > The RocksDBFlowFileRepository has a "Recovery Mode" feature which limits the > number of FlowFiles that are loaded into the system and processed before more > FlowFiles are loaded in. This is useful when Nifi fails to start on a system > with insufficient memory. > The same feature should be added to the WriteAheadFlowFileRepository as the > RocksDBFlowFileRepository is depricated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12210) Implement recovery mode for WriteAheadFlowFileRepository
Reynaldo Rea created NIFI-12210: --- Summary: Implement recovery mode for WriteAheadFlowFileRepository Key: NIFI-12210 URL: https://issues.apache.org/jira/browse/NIFI-12210 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Reynaldo Rea The RocksDBFlowFileRepository has a "Recovery Mode" feature which limits the number of FlowFiles that are loaded into the system and processed before more FlowFiles are loaded in. This is useful when Nifi fails to start on a system with insufficient memory. The same feature should be added to the WriteAheadFlowFileRepository as the RocksDBFlowFileRepository is depricated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12174: Upgrading AWS Lambda processor to SDK 2.x [nifi]
gresockj commented on code in PR #7851:
URL: https://github.com/apache/nifi/pull/7851#discussion_r1355097793
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/lambda/PutLambda.java:
##
@@ -155,33 +150,38 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
return;
}
-final AWSLambdaClient client = getClient(context);
+final LambdaClient client = getClient(context);
try {
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
session.exportTo(flowFile, baos);
Review Comment:
Good catch -- thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12207) Upgrade Netty to 4.1.100
[ https://issues.apache.org/jira/browse/NIFI-12207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12207: Status: Patch Available (was: Open) > Upgrade Netty to 4.1.100 > > > Key: NIFI-12207 > URL: https://issues.apache.org/jira/browse/NIFI-12207 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: dependency-upgrade > Fix For: 2.0.0, 1.24.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to 4.1.100 to mitigate > [CVE-2023-44487|https://www.cve.org/CVERecord?id=CVE-2023-44487], related to > potential Denial of Service for server components supporting HTTP/2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12208) Upgrade Jetty to 10.0.17
[ https://issues.apache.org/jira/browse/NIFI-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12208: Status: Patch Available (was: Open) > Upgrade Jetty to 10.0.17 > > > Key: NIFI-12208 > URL: https://issues.apache.org/jira/browse/NIFI-12208 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Jetty dependencies on the main branch should be upgraded to > [10.0.17|https://github.com/eclipse/jetty.project/releases/tag/jetty-10.0.17] > to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12209) Upgrade Jetty to 9.4.53 on Support Branch
[ https://issues.apache.org/jira/browse/NIFI-12209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12209: Status: Patch Available (was: Open) > Upgrade Jetty to 9.4.53 on Support Branch > - > > Key: NIFI-12209 > URL: https://issues.apache.org/jira/browse/NIFI-12209 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: dependency-upgrade > Fix For: 1.24.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Jetty dependencies on the support branch should be upgraded to > [9.4.53|https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.53.v20231009] > to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12209 Upgrade Jetty from 9.4.52 to 9.4.53 [nifi]
exceptionfactory opened a new pull request, #7867: URL: https://github.com/apache/nifi/pull/7867 # Summary [NIFI-12209](https://issues.apache.org/jira/browse/NIFI-12209) Upgrades Jetty dependencies from 9.4.52 to [9.4.53](https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.53.v20231009) on the support branch in order to mitigate potential Denial of Service attacks related to HTTP/2 reset frame handling. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12208 Upgrade Jetty from 10.0.16 to 10.0.17 [nifi]
exceptionfactory opened a new pull request, #7866: URL: https://github.com/apache/nifi/pull/7866 # Summary [NIFI-12208](https://issues.apache.org/jira/browse/NIFI-12208) Upgrades Jetty dependencies from 10.0.16 to [10.0.17](https://github.com/eclipse/jetty.project/releases/tag/jetty-10.0.17) in order to mitigate potential Denial of Service attacks related to HTTP/2 reset frame handling. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12209) Upgrade Jetty to 9.4.53 on Support Branch
David Handermann created NIFI-12209: --- Summary: Upgrade Jetty to 9.4.53 on Support Branch Key: NIFI-12209 URL: https://issues.apache.org/jira/browse/NIFI-12209 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 1.24.0 Jetty dependencies on the support branch should be upgraded to [9.4.53|https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.53.v20231009] to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12208) Upgrade Jetty to 10.0.17
David Handermann created NIFI-12208: --- Summary: Upgrade Jetty to 10.0.17 Key: NIFI-12208 URL: https://issues.apache.org/jira/browse/NIFI-12208 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0 Jetty dependencies on the main branch should be upgraded to [10.0.17|https://github.com/eclipse/jetty.project/releases/tag/jetty-10.0.17] to mitigate potential vulnerabilities related to HTTP/2 reset frame handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12207 Upgrade Netty from 4.1.99 to 4.1.100 [nifi]
exceptionfactory opened a new pull request, #7865: URL: https://github.com/apache/nifi/pull/7865 # Summary [NIFI-12207](https://issues.apache.org/jira/browse/NIFI-12207) Upgrades Netty libraries from 4.1.99 to [4.1.100](https://netty.io/news/2023/10/10/4-1-100-Final.html) in order to mitigate CVE-2023-44487 related to potential attacks on server components supporting HTTP/2. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12207) Upgrade Netty to 4.1.100
David Handermann created NIFI-12207: --- Summary: Upgrade Netty to 4.1.100 Key: NIFI-12207 URL: https://issues.apache.org/jira/browse/NIFI-12207 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0, 1.24.0 Netty dependencies should be upgraded to 4.1.100 to mitigate [CVE-2023-44487|https://www.cve.org/CVERecord?id=CVE-2023-44487], related to potential Denial of Service for server components supporting HTTP/2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2239 Add bandwidth limit to InvokeHTTP processor [nifi-minifi-cpp]
lordgamez commented on code in PR #1674:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1674#discussion_r1355003290
##
libminifi/include/core/PropertyType.h:
##
@@ -319,6 +319,24 @@ class TimePeriodPropertyType : public PropertyType {
}
};
+class DataTransferSpeedPropertyType : public PropertyType {
+ public:
+ constexpr ~DataTransferSpeedPropertyType() override {} // NOLINT see
comment at parent
+
+ [[nodiscard]] std::string_view getValidatorName() const override { return
"DATA_SIZE_VALIDATOR"; }
Review Comment:
Changed to `DATA_TRANSFER_SPEED_VALIDATOR` in
f8833c3b01c5edf0ff60fff593b2f21ab91763cd, it should be ignored on C2 server
side if not implemented.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2239 Add bandwidth limit to InvokeHTTP processor [nifi-minifi-cpp]
lordgamez commented on code in PR #1674:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1674#discussion_r1355001874
##
PROCESSORS.md:
##
@@ -1320,6 +1320,8 @@ In the list below, the names of required properties
appear in bold. Any other pr
| Always Output Response | false|
truefalse | Will force a response FlowFile to be generated
and routed to the 'Response' relationship regardless of what the server status
code received is
|
| Penalize on "No Retry" | false|
truefalse | Enabling this property will penalize FlowFiles
that are routed to the "No Retry" relationship.
|
| **Invalid HTTP Header Field Handling Strategy** | transform|
failtransformdrop | Indicates what should happen when an attribute's
name is not a valid HTTP header field name. Options: transform - invalid
characters are replaced, fail - flow file is transferred to failure, drop -
drops invalid attributes from HTTP message
|
+| **Upload Speed Limit** | 0 B/s|
| Maximum data per second to send, use 0 B/s for
unlimited.
|
+| **Download Speed Limit**| 0 B/s|
| Maximum data per second to receive, use 0 B/s for
unlimited.
|
Review Comment:
Updated in f8833c3b01c5edf0ff60fff593b2f21ab91763cd
##
libminifi/include/core/TypedValues.h:
##
@@ -149,10 +144,39 @@ class DataSizeValue : public TransformableValue, public
state::response::UInt64V
}
};
-} // namespace core
-} // namespace minifi
-} // namespace nifi
-} // namespace apache
-} // namespace org
+class DataTransferSpeedValue : public TransformableValue, public
state::response::UInt64Value {
+ public:
+ MINIFIAPI static const std::type_index type_id;
+
+ explicit DataTransferSpeedValue(const std::string _speed_string)
+ : state::response::UInt64Value(0) {
+
DataSizeValue::StringToInt(removePerSecSuffix(transfer_speed_string),
value);
+string_value = transfer_speed_string;
+ }
+
+ explicit DataTransferSpeedValue(uint64_t value)
+ : state::response::UInt64Value(value) {
+ }
+
+ DataTransferSpeedValue()
+ : state::response::UInt64Value(0) {
+ }
+
+ static std::string removePerSecSuffix(const std::string ) {
+auto lower_case_input = utils::StringUtils::toLower(input);
+if (lower_case_input.ends_with("/sec")) {
+ return input.substr(0, input.size() - 4);
+}
Review Comment:
Updated in f8833c3b01c5edf0ff60fff593b2f21ab91763cd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11789) ExecuteSQL doesn't set fragment.count attribute when Output Batch Size is set
[ https://issues.apache.org/jira/browse/NIFI-11789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774055#comment-17774055 ] Matt Burgess commented on NIFI-11789: - That is correct. Although the PR fixes your use case, I believe it would take a significant refactor in order to keep track of the FlowFile "groups" such that we know how many FlowFiles are in the group by the time the session gets committed (versus the different points in code where the FlowFiles are transferred within the session, they are not sent until the session is committed). For your use case it was more straightforward, but there are cases depending on the configuration of the processor where different FlowFiles are transferred at different points and the session is committed later. After the FlowFile has been transferred it cannot be altered (i.e. the fragment.count attribute cannot be set) If someone wants to start with my PR and has a better approach, or wants to start from scratch, I encourage them to do so. Also if I feel like revisiting it with fresh eyes down the road, I may do that. As a workaround / solution, if the outgoing FlowFiles have the correct number of records in them (meaning the record count is accurate for what the fragment.count attribute SHOULD be and the fragment.index attribute values are correct), you can pass the FlowFiles through a CalculateRecordStats processor then UpdateAttribute to set "fragment.count" to the value of "record.count", and the merge should work downstream. > ExecuteSQL doesn't set fragment.count attribute when Output Batch Size is set > - > > Key: NIFI-11789 > URL: https://issues.apache.org/jira/browse/NIFI-11789 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Tamas Neumer >Assignee: Matt Burgess >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Hi, > I am working with the ExecuteSQL processor and discovered an unexpected > behavior. If I specify the attribute "Output Batch Size", I get the > fragment.index on the outflowing flowing Flowfiles, but the fragment.count > attribute is not set (according to the documentation). > The behavior I would expect (in line with how merge processors work) is that > the attribute fragment.count is just set at the last Flowfile for the batch. > This would make it possible to merge all the batches together afterward. > So my proposal, in short, is that the fragment.count should be set in the > last Flowfile of a batch. > BR Florian -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2225) Improve logging in PutFile/PutSmb
[ https://issues.apache.org/jira/browse/MINIFICPP-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2225: --- Status: Patch Available (was: In Progress) > Improve logging in PutFile/PutSmb > - > > Key: MINIFICPP-2225 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2225 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > We are warning for normal operation which could lead to spamming to logs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2239 Add bandwidth limit to InvokeHTTP processor [nifi-minifi-cpp]
szaszm commented on code in PR #1674:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1674#discussion_r1354915765
##
libminifi/include/core/TypedValues.h:
##
@@ -149,10 +144,39 @@ class DataSizeValue : public TransformableValue, public
state::response::UInt64V
}
};
-} // namespace core
-} // namespace minifi
-} // namespace nifi
-} // namespace apache
-} // namespace org
+class DataTransferSpeedValue : public TransformableValue, public
state::response::UInt64Value {
+ public:
+ MINIFIAPI static const std::type_index type_id;
+
+ explicit DataTransferSpeedValue(const std::string _speed_string)
+ : state::response::UInt64Value(0) {
+
DataSizeValue::StringToInt(removePerSecSuffix(transfer_speed_string),
value);
+string_value = transfer_speed_string;
+ }
+
+ explicit DataTransferSpeedValue(uint64_t value)
+ : state::response::UInt64Value(value) {
+ }
+
+ DataTransferSpeedValue()
+ : state::response::UInt64Value(0) {
+ }
+
+ static std::string removePerSecSuffix(const std::string ) {
+auto lower_case_input = utils::StringUtils::toLower(input);
+if (lower_case_input.ends_with("/sec")) {
+ return input.substr(0, input.size() - 4);
+}
Review Comment:
I'd prefer to keep the set of accepted formats smaller, and only accept
"/s", not "/sec" or "ps". It's easy to extend later, but it harder to break
what worked before.
##
PROCESSORS.md:
##
@@ -1320,6 +1320,8 @@ In the list below, the names of required properties
appear in bold. Any other pr
| Always Output Response | false|
truefalse | Will force a response FlowFile to be generated
and routed to the 'Response' relationship regardless of what the server status
code received is
|
| Penalize on "No Retry" | false|
truefalse | Enabling this property will penalize FlowFiles
that are routed to the "No Retry" relationship.
|
| **Invalid HTTP Header Field Handling Strategy** | transform|
failtransformdrop | Indicates what should happen when an attribute's
name is not a valid HTTP header field name. Options: transform - invalid
characters are replaced, fail - flow file is transferred to failure, drop -
drops invalid attributes from HTTP message
|
+| **Upload Speed Limit** | 0 B/s|
| Maximum data per second to send, use 0 B/s for
unlimited.
|
+| **Download Speed Limit**| 0 B/s|
| Maximum data per second to receive, use 0 B/s for
unlimited.
|
Review Comment:
It would be better to treat an empty value as no limit. We can also keep
treating "0 B/s" as no limit, but I'd document the empty as unlimited instead.
```suggestion
| **Upload Speed Limit** | 0 B/s
| | Maximum data per second to send. Leave this
empty if you want no limit.
|
| **Download Speed Limit**| 0 B/s
| | Maximum data per second to receive. Leave this
empty if you want no limit.
|
```
##
libminifi/include/core/PropertyType.h:
##
@@ -319,6 +319,24 @@ class TimePeriodPropertyType : public PropertyType {
}
};
+class DataTransferSpeedPropertyType : public PropertyType {
+ public:
+ constexpr ~DataTransferSpeedPropertyType() override {}
[PR] MINIFICPP-2225 Improve logging in PutFile/PutSmb [nifi-minifi-cpp]
martinzink opened a new pull request, #1678: URL: https://github.com/apache/nifi-minifi-cpp/pull/1678 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11789) ExecuteSQL doesn't set fragment.count attribute when Output Batch Size is set
[ https://issues.apache.org/jira/browse/NIFI-11789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11789: Fix Version/s: (was: 1.latest) (was: 2.latest) > ExecuteSQL doesn't set fragment.count attribute when Output Batch Size is set > - > > Key: NIFI-11789 > URL: https://issues.apache.org/jira/browse/NIFI-11789 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Tamas Neumer >Assignee: Matt Burgess >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Hi, > I am working with the ExecuteSQL processor and discovered an unexpected > behavior. If I specify the attribute "Output Batch Size", I get the > fragment.index on the outflowing flowing Flowfiles, but the fragment.count > attribute is not set (according to the documentation). > The behavior I would expect (in line with how merge processors work) is that > the attribute fragment.count is just set at the last Flowfile for the batch. > This would make it possible to merge all the batches together afterward. > So my proposal, in short, is that the fragment.count should be set in the > last Flowfile of a batch. > BR Florian -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2225) Improve logging in PutFile/PutSmb
[ https://issues.apache.org/jira/browse/MINIFICPP-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2225: --- Priority: Trivial (was: Major) > Improve logging in PutFile/PutSmb > - > > Key: MINIFICPP-2225 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2225 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Trivial > > We are warning for normal operation which could lead to spamming to logs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2246) Update UVC to fix macOS compilation
[ https://issues.apache.org/jira/browse/MINIFICPP-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2246: --- Status: Patch Available (was: In Progress) > Update UVC to fix macOS compilation > --- > > Key: MINIFICPP-2246 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2246 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > USB extension doesnt compile on macOS -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] MINIFICPP-2246 Update libuvc [nifi-minifi-cpp]
martinzink opened a new pull request, #1677: URL: https://github.com/apache/nifi-minifi-cpp/pull/1677 This fixes a compilation error on macOS (previously couldnt find libuvc) --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-8135 allow CHOICE data types in conversion of Records to Java Maps [nifi]
ChrisSamo632 commented on code in PR #7746:
URL: https://github.com/apache/nifi/pull/7746#discussion_r1354608522
##
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java:
##
@@ -854,79 +855,82 @@ public static Map toMap(final Object
value, final String fieldNa
*/
@SuppressWarnings({"unchecked", "rawtypes"})
public static Object convertRecordFieldtoObject(final Object value, final
DataType dataType) {
-
if (value == null) {
return null;
}
+DataType chosenDataType;
+if (dataType instanceof ChoiceDataType) {
+final DataType chosen = chooseDataType(value, (ChoiceDataType)
dataType);
+chosenDataType = chosen != null ? chosen : dataType;
+} else {
+chosenDataType = dataType;
+}
+
if (value instanceof Record) {
-Record record = (Record) value;
-RecordSchema recordSchema = record.getSchema();
+final Record record = (Record) value;
+final RecordSchema recordSchema = record.getSchema();
if (recordSchema == null) {
throw new IllegalTypeConversionException("Cannot convert value
of type Record to Map because Record does not have an associated Schema");
}
-final Map recordMap = new LinkedHashMap<>();
-for (RecordField field : recordSchema.getFields()) {
-final DataType fieldDataType = field.getDataType();
+final Map recordMap = new
LinkedHashMap<>(record.getRawFieldNames().size(), 1);
+for (final RecordField field : recordSchema.getFields()) {
+if (field.getDataType() instanceof ChoiceDataType) {
+final DataType chosen = chooseDataType(value,
(ChoiceDataType) field.getDataType());
+chosenDataType = chosen != null ? chosen :
field.getDataType();
+} else {
+chosenDataType = field.getDataType();
+}
final String fieldName = field.getFieldName();
-Object fieldValue = record.getValue(fieldName);
+final Object fieldValue = record.getValue(fieldName);
if (fieldValue == null) {
recordMap.put(fieldName, null);
-} else if (isScalarValue(fieldDataType, fieldValue)) {
+} else if (isScalarValue(chosenDataType, fieldValue)) {
recordMap.put(fieldName, fieldValue);
-} else if (fieldDataType instanceof RecordDataType) {
+} else if (chosenDataType instanceof RecordDataType) {
Record nestedRecord = (Record) fieldValue;
Review Comment:
Recent example from the [NiFi Dev Mailing
List](https://lists.apache.org/thread/kcnsxvbbdfwfhj0tdsyn53x8ljhgdt1v)
containing some failing data and associated Jolt Transform Spec.
Currently fails on this branch with:
```
java.lang.ClassCastException: class java.lang.String cannot be cast to class
org.apache.nifi.serialization.record.Record (java.lang.String is in module
java.base of loader 'bootstrap'; org.apache.nifi.serialization.record.Record is
in unnamed module of loader org.apache.nifi.nar.NarClassLoader @4b3ad7ca)
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertRecordFieldtoObject(DataTypeUtils.java:893)
at
org.apache.nifi.processors.jolt.record.JoltTransformRecord.transform(JoltTransformRecord.java:425)
```
Fails on `main`/`support/nifi-1.x` with:
```
Cannot convert value [MapRecord[{A: 192,B: 32,C: 13,D: 19}]] of
CHOICE[STRING,RECORD] to Map for field dimensions because type is not
supported.
```
So this change for NIFI-8135 has at least had an impact, albeit not quite
resolved the problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2226 Fix LoggerTests flakiness [nifi-minifi-cpp]
szaszm commented on code in PR #1676:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1676#discussion_r1354588041
##
libminifi/include/core/logging/internal/LogCompressorSink.h:
##
@@ -54,7 +48,7 @@ class LogCompressorSink : public
spdlog::sinks::base_sink {
void flush_() override;
public:
- explicit LogCompressorSink(LogQueueSize cache_size, LogQueueSize
compressed_size, std::shared_ptr logger);
+ LogCompressorSink(LogQueueSize cache_size, LogQueueSize compressed_size,
const std::shared_ptr& logger);
Review Comment:
Makes sense. It's fine as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (MINIFICPP-2246) Update UVC to fix macOS compilation
Martin Zink created MINIFICPP-2246: -- Summary: Update UVC to fix macOS compilation Key: MINIFICPP-2246 URL: https://issues.apache.org/jira/browse/MINIFICPP-2246 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Martin Zink Assignee: Martin Zink USB extension doesnt compile on macOS -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2226 Fix LoggerTests flakiness [nifi-minifi-cpp]
fgerlits commented on code in PR #1676:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1676#discussion_r1354422474
##
libminifi/include/core/logging/internal/LogCompressorSink.h:
##
@@ -54,7 +48,7 @@ class LogCompressorSink : public
spdlog::sinks::base_sink {
void flush_() override;
public:
- explicit LogCompressorSink(LogQueueSize cache_size, LogQueueSize
compressed_size, std::shared_ptr logger);
+ LogCompressorSink(LogQueueSize cache_size, LogQueueSize compressed_size,
const std::shared_ptr& logger);
Review Comment:
The detail you are missing is that the `logger` parameter is used twice in
the constructor: it is passed to the `ActiveCompressor::Allocator` in the
`compressed_logs_` field, as well as stored directly in the
`compressor_logger_` field. I did not like the looks of passing by value,
copying once and then moving. I can change it to that if you prefer it.
I removed `explicit` because it doesn't make much sense for constructors
with multiple parameters, but I can put it back if you think it's useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2226 Fix LoggerTests flakiness [nifi-minifi-cpp]
szaszm commented on code in PR #1676:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1676#discussion_r1354270590
##
libminifi/include/core/logging/internal/LogCompressorSink.h:
##
@@ -54,7 +48,7 @@ class LogCompressorSink : public
spdlog::sinks::base_sink {
void flush_() override;
public:
- explicit LogCompressorSink(LogQueueSize cache_size, LogQueueSize
compressed_size, std::shared_ptr logger);
+ LogCompressorSink(LogQueueSize cache_size, LogQueueSize compressed_size,
const std::shared_ptr& logger);
Review Comment:
I don't think passing shared_ptrs by const ref is normally a good idea. If
it takes ownership (like in this case), passing by value (and move to data
member in this case) is probably best. If it's observer only, then passing a
raw pointer or reference usually works better. Am I missing an important detail?
Also, is there any reason to remove explicit? I don't have strong
preferences either way for constructors with multiple parameters, I just don't
understand why it's changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12140) Possibility to configure ACLs for redis connections
[ https://issues.apache.org/jira/browse/NIFI-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773928#comment-17773928 ] Fabian Reiber commented on NIFI-12140: -- Thanks for fast implementation :) > Possibility to configure ACLs for redis connections > --- > > Key: NIFI-12140 > URL: https://issues.apache.org/jira/browse/NIFI-12140 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.21.0 >Reporter: Fabian Reiber >Assignee: Bryan Bende >Priority: Major > Fix For: 2.0.0, 1.24.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The RedisConnectionPoolService and RedisDistributedMapCacheClientService do > not allow to configure acls. At least I can't find it. Is this a feature to > be added in near future? > The current possibility to configure a password in RedisConnectionPoolService > reflects the "requirepass" in redis configuration. It also means to enable > acls configuration for redis sentinel as well. And maybe for redis cluster > too (I don't know, if redis cluster uses different acls as well)? > I think it makes sense to enable this feature for > RedisDistributedMapCacheClientService. > What do you think about it? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] MINIFICPP-2226 Fix LoggerTests flakiness [nifi-minifi-cpp]
fgerlits opened a new pull request, #1676: URL: https://github.com/apache/nifi-minifi-cpp/pull/1676 https://issues.apache.org/jira/browse/MINIFICPP-2226 - don't use the static LoggerConfiguration instance, use our own instance instead - do not call compress(), as that will conflict with the compress() calls in the background thread; wait for the background thread to finish instead - add mutex to compress(), as it it called both from `run()` and `getContent()` in `LogCompressorSink`, and these are on different threads - add `LogTestController::getInstance().clear()` calls to isolate tests from each other - use different logger names in each test, because reusing the same name causes problems - add a missing `return false` in LogCompressorSink.cpp --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
