[GitHub] [nifi] vitalyzhakov opened a new pull request, #6329: NIFI-10360 VOLUME declaration prevents updating flow from CI
vitalyzhakov opened a new pull request, #6329: URL: https://github.com/apache/nifi/pull/6329 # Summary [NIFI-10360](https://issues.apache.org/jira/browse/NIFI-10360) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10390) The request contained an invalid host header [xx.xxx.xx.xx9443] in the request [/nifi]. Check for request manipulation or third-party intercept. Valid host headers are [e
[
https://issues.apache.org/jira/browse/NIFI-10390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lijunbao updated NIFI-10390:
Description:
{{docker run --name nifi \
-p 9443:9443 \
-d \
-e NIFI_WEB_HTTPS_PORT='9443' \
apache/nifi:latest}}
{{The request contained an invalid host header [xx.xxx.xx.xx9443] in the
request [/nifi]. Check for request manipulation or third-party intercept. Valid
host headers are [empty] or:}}
{{}}
* 127.0.0.1
* 127.0.0.1:8443
* localhost
* localhost:8443
* [::1]
* [::1]:8443
* b0a691ba82f9
* b0a691ba82f9:8443
* 172.17.0.15
* 172.17.0.15:8443
* 147.128.16.5
There is an error in docker deployment of nifi. This is a magic problem. HTTPS
verifies how to use docker for account login and deploys docker on dockerhub.
Please ask the boss for advice
{{}}
was:
{{docker run --name nifi \
-p 9443:9443 \
-d \
-e NIFI_WEB_HTTPS_PORT='9443' \
apache/nifi:latest}}
{{The request contained an invalid host header [xx.xxx.xx.xx8443] in the
request [/nifi]. Check for request manipulation or third-party intercept. Valid
host headers are [empty] or:}}
{{}}
* 127.0.0.1
* 127.0.0.1:8443
* localhost
* localhost:8443
* [::1]
* [::1]:8443
* b0a691ba82f9
* b0a691ba82f9:8443
* 172.17.0.15
* 172.17.0.15:8443
* 147.128.16.5
There is an error in docker deployment of nifi. This is a magic problem. HTTPS
verifies how to use docker for account login and deploys docker on dockerhub.
Please ask the boss for advice
{{}}
> The request contained an invalid host header [xx.xxx.xx.xx9443] in the
> request [/nifi]. Check for request manipulation or third-party intercept.
> Valid host headers are [empty] or:
> ---
>
> Key: NIFI-10390
> URL: https://issues.apache.org/jira/browse/NIFI-10390
> Project: Apache NiFi
> Issue Type: Bug
> Components: Docker
>Affects Versions: 1.16.3
> Environment: Docker deployment nifi
>Reporter: lijunbao
>Priority: Major
> Fix For: 1.16.3
>
>
>
> {{docker run --name nifi \
> -p 9443:9443 \
> -d \
> -e NIFI_WEB_HTTPS_PORT='9443' \
> apache/nifi:latest}}
> {{The request contained an invalid host header [xx.xxx.xx.xx9443] in the
> request [/nifi]. Check for request manipulation or third-party intercept.
> Valid host headers are [empty] or:}}
> {{}}
> * 127.0.0.1
> * 127.0.0.1:8443
> * localhost
> * localhost:8443
> * [::1]
> * [::1]:8443
> * b0a691ba82f9
> * b0a691ba82f9:8443
> * 172.17.0.15
> * 172.17.0.15:8443
> * 147.128.16.5
>
> There is an error in docker deployment of nifi. This is a magic problem.
> HTTPS verifies how to use docker for account login and deploys docker on
> dockerhub. Please ask the boss for advice
>
> {{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10390) The request contained an invalid host header [xx.xxx.xx.xx9443] in the request [/nifi]. Check for request manipulation or third-party intercept. Valid host headers are [e
[
https://issues.apache.org/jira/browse/NIFI-10390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lijunbao updated NIFI-10390:
Description:
{{docker run --name nifi \
-p 9443:9443 \
-d \
-e NIFI_WEB_HTTPS_PORT='9443' \
apache/nifi:latest}}
{{The request contained an invalid host header [xx.xxx.xx.xx8443] in the
request [/nifi]. Check for request manipulation or third-party intercept. Valid
host headers are [empty] or:}}
{{}}
* 127.0.0.1
* 127.0.0.1:8443
* localhost
* localhost:8443
* [::1]
* [::1]:8443
* b0a691ba82f9
* b0a691ba82f9:8443
* 172.17.0.15
* 172.17.0.15:8443
* 147.128.16.5
There is an error in docker deployment of nifi. This is a magic problem. HTTPS
verifies how to use docker for account login and deploys docker on dockerhub.
Please ask the boss for advice
{{}}
was:
{{docker run --name nifi \
-p 9443:9443 \
-d \
-e NIFI_WEB_HTTPS_PORT='9443' \
apache/nifi:latest}}
{{The request contained an invalid host header [xx.xxx.xx.xx8443] in the
request [/nifi]. Check for request manipulation or third-party intercept. Valid
host headers are [empty] or:}}
{{}}
* 127.0.0.1
* 127.0.0.1:8443
* localhost
* localhost:8443
* [::1]
* [::1]:8443
* b0a691ba82f9
* b0a691ba82f9:8443
* 172.17.0.15
* 172.17.0.15:8443
* 147.128.16.5
There is an error in docker deployment of nifi. This is a magic problem. HTTPS
verifies how to use docker for account login and deploys docker on dockerhub.
Please ask the boss for advice
{{}}
Summary: The request contained an invalid host header
[xx.xxx.xx.xx9443] in the request [/nifi]. Check for request manipulation or
third-party intercept. Valid host headers are [empty] or: (was: The request
contained an invalid host header [xx.xxx.xx.xx8443] in the request [/nifi].
Check for request manipulation or third-party intercept. Valid host headers are
[empty] or:)
> The request contained an invalid host header [xx.xxx.xx.xx9443] in the
> request [/nifi]. Check for request manipulation or third-party intercept.
> Valid host headers are [empty] or:
> ---
>
> Key: NIFI-10390
> URL: https://issues.apache.org/jira/browse/NIFI-10390
> Project: Apache NiFi
> Issue Type: Bug
> Components: Docker
>Affects Versions: 1.16.3
> Environment: Docker deployment nifi
>Reporter: lijunbao
>Priority: Major
> Fix For: 1.16.3
>
>
>
> {{docker run --name nifi \
> -p 9443:9443 \
> -d \
> -e NIFI_WEB_HTTPS_PORT='9443' \
> apache/nifi:latest}}
> {{The request contained an invalid host header [xx.xxx.xx.xx8443] in the
> request [/nifi]. Check for request manipulation or third-party intercept.
> Valid host headers are [empty] or:}}
> {{}}
> * 127.0.0.1
> * 127.0.0.1:8443
> * localhost
> * localhost:8443
> * [::1]
> * [::1]:8443
> * b0a691ba82f9
> * b0a691ba82f9:8443
> * 172.17.0.15
> * 172.17.0.15:8443
> * 147.128.16.5
>
> There is an error in docker deployment of nifi. This is a magic problem.
> HTTPS verifies how to use docker for account login and deploys docker on
> dockerhub. Please ask the boss for advice
>
> {{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-10390) The request contained an invalid host header [xx.xxx.xx.xx8443] in the request [/nifi]. Check for request manipulation or third-party intercept. Valid host headers are [e
lijunbao created NIFI-10390:
---
Summary: The request contained an invalid host header
[xx.xxx.xx.xx8443] in the request [/nifi]. Check for request manipulation or
third-party intercept. Valid host headers are [empty] or:
Key: NIFI-10390
URL: https://issues.apache.org/jira/browse/NIFI-10390
Project: Apache NiFi
Issue Type: Bug
Components: Docker
Affects Versions: 1.16.3
Environment: Docker deployment nifi
Reporter: lijunbao
Fix For: 1.16.3
{{docker run --name nifi \
-p 9443:9443 \
-d \
-e NIFI_WEB_HTTPS_PORT='9443' \
apache/nifi:latest}}
{{The request contained an invalid host header [xx.xxx.xx.xx8443] in the
request [/nifi]. Check for request manipulation or third-party intercept. Valid
host headers are [empty] or:}}
{{}}
* 127.0.0.1
* 127.0.0.1:8443
* localhost
* localhost:8443
* [::1]
* [::1]:8443
* b0a691ba82f9
* b0a691ba82f9:8443
* 172.17.0.15
* 172.17.0.15:8443
* 147.128.16.5
There is an error in docker deployment of nifi. This is a magic problem. HTTPS
verifies how to use docker for account login and deploys docker on dockerhub.
Please ask the boss for advice
{{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] nandorsoma commented on pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on PR #6225: URL: https://github.com/apache/nifi/pull/6225#issuecomment-1225043867 Rebased on top of current main, that's why the force push. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r953196811
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/AbstractMQTTProcessor.java:
##
@@ -289,88 +288,35 @@ public Collection customValidate(final
ValidationContext valid
return results;
}
-public static Properties transformSSLContextService(SSLContextService
sslContextService){
-Properties properties = new Properties();
-if (sslContextService.getSslAlgorithm() != null) {
-properties.setProperty("com.ibm.ssl.protocol",
sslContextService.getSslAlgorithm());
-}
-if (sslContextService.getKeyStoreFile() != null) {
-properties.setProperty("com.ibm.ssl.keyStore",
sslContextService.getKeyStoreFile());
-}
-if (sslContextService.getKeyStorePassword() != null) {
-properties.setProperty("com.ibm.ssl.keyStorePassword",
sslContextService.getKeyStorePassword());
-}
-if (sslContextService.getKeyStoreType() != null) {
-properties.setProperty("com.ibm.ssl.keyStoreType",
sslContextService.getKeyStoreType());
-}
-if (sslContextService.getTrustStoreFile() != null) {
-properties.setProperty("com.ibm.ssl.trustStore",
sslContextService.getTrustStoreFile());
-}
-if (sslContextService.getTrustStorePassword() != null) {
-properties.setProperty("com.ibm.ssl.trustStorePassword",
sslContextService.getTrustStorePassword());
-}
-if (sslContextService.getTrustStoreType() != null) {
-properties.setProperty("com.ibm.ssl.trustStoreType",
sslContextService.getTrustStoreType());
-}
-return properties;
-}
-
-protected void onScheduled(final ProcessContext context){
-broker =
context.getProperty(PROP_BROKER_URI).evaluateAttributeExpressions().getValue();
-brokerUri = broker.endsWith("/") ? broker : broker + "/";
-clientID =
context.getProperty(PROP_CLIENTID).evaluateAttributeExpressions().getValue();
-
-if (clientID == null) {
-clientID = UUID.randomUUID().toString();
-}
-
-connOpts = new MqttConnectOptions();
-
connOpts.setCleanSession(context.getProperty(PROP_CLEAN_SESSION).asBoolean());
-
connOpts.setKeepAliveInterval(context.getProperty(PROP_KEEP_ALIVE_INTERVAL).asInteger());
-
connOpts.setMqttVersion(context.getProperty(PROP_MQTT_VERSION).asInteger());
-
connOpts.setConnectionTimeout(context.getProperty(PROP_CONN_TIMEOUT).asInteger());
-
-PropertyValue sslProp = context.getProperty(PROP_SSL_CONTEXT_SERVICE);
-if (sslProp.isSet()) {
-Properties sslProps =
transformSSLContextService((SSLContextService) sslProp.asControllerService());
-connOpts.setSSLProperties(sslProps);
-}
-
-PropertyValue lastWillTopicProp =
context.getProperty(PROP_LAST_WILL_TOPIC);
-if (lastWillTopicProp.isSet()){
-String lastWillMessage =
context.getProperty(PROP_LAST_WILL_MESSAGE).getValue();
-PropertyValue lastWillRetain =
context.getProperty(PROP_LAST_WILL_RETAIN);
-Integer lastWillQOS =
context.getProperty(PROP_LAST_WILL_QOS).asInteger();
-connOpts.setWill(lastWillTopicProp.getValue(),
lastWillMessage.getBytes(), lastWillQOS, lastWillRetain.isSet() ?
lastWillRetain.asBoolean() : false);
-}
-
-
-PropertyValue usernameProp = context.getProperty(PROP_USERNAME);
-if(usernameProp.isSet()) {
-
connOpts.setUserName(usernameProp.evaluateAttributeExpressions().getValue());
-
connOpts.setPassword(context.getProperty(PROP_PASSWORD).getValue().toCharArray());
-}
+protected void onScheduled(final ProcessContext context) {
+clientProperties = getMqttClientProperties(context);
+connectionProperties = getMqttConnectionProperties(context);
}
protected void onStopped() {
-try {
-logger.info("Disconnecting client");
-mqttClient.disconnect(DISCONNECT_TIMEOUT);
-} catch(MqttException me) {
-logger.error("Error disconnecting MQTT client due to {}", new
Object[]{me.getMessage()}, me);
-}
+// Since client is created in the onTrigger method it can happen that
it never will be created because of an initialization error.
+// We are preventing additional nullPtrException here, but the clean
solution would be to create the client in the onScheduled method.
+if (mqttClient != null) {
+try {
+logger.info("Disconnecting client");
+mqttClient.disconnect(DISCONNECT_TIMEOUT);
+} catch (MqttException me) {
+logger.error("Error disconnecting MQTT client due to {}", new
Object[]{me.getMessage()}, me);
+}
+
+
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6273: NIFI-9953 - The config encryption tool is too complicated to use and can be simplified
exceptionfactory commented on code in PR #6273:
URL: https://github.com/apache/nifi/pull/6273#discussion_r953194995
##
nifi-toolkit/nifi-property-encryptor-tool/src/main/java/org/apache/nifi/xml/XmlCryptoParser.java:
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.xml;
+
+import org.apache.nifi.properties.SensitivePropertyProvider;
+import org.apache.nifi.properties.SensitivePropertyProviderFactory;
+import org.apache.nifi.properties.scheme.ProtectionScheme;
+
+import javax.xml.namespace.QName;
+import javax.xml.stream.XMLEventFactory;
+import javax.xml.stream.XMLEventReader;
+import javax.xml.stream.XMLEventWriter;
+import javax.xml.stream.XMLInputFactory;
+import javax.xml.stream.XMLOutputFactory;
+import javax.xml.stream.XMLStreamException;
+import javax.xml.stream.events.Attribute;
+import javax.xml.stream.events.Characters;
+import javax.xml.stream.events.StartElement;
+import javax.xml.stream.events.XMLEvent;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.Objects;
+
+public abstract class XmlCryptoParser {
+
+protected static final String ENCRYPTION_ATTRIBUTE_NAME = "encryption";
+protected static final String PARENT_IDENTIFIER = "identifier";
+protected static final String PROPERTY_ELEMENT = "property";
+
+protected final SensitivePropertyProvider cryptoProvider;
+protected SensitivePropertyProviderFactory providerFactory;
+
+public XmlCryptoParser(final SensitivePropertyProviderFactory
providerFactory, final ProtectionScheme scheme) {
+this.providerFactory = providerFactory;
+cryptoProvider = providerFactory.getProvider(scheme);
+}
+
+protected void cryptographicXmlOperation(final InputStream
encryptedXmlContent, final OutputStream decryptedOutputStream) {
+final XMLOutputFactory factory = XMLOutputFactory.newInstance();
+factory.setProperty("com.ctc.wstx.outputValidateStructure", false);
+
+try {
+final XMLEventReader eventReader =
getXMLReader(encryptedXmlContent);
+final XMLEventWriter xmlWriter =
factory.createXMLEventWriter(decryptedOutputStream);
+String groupIdentifier = "";
+
+while(eventReader.hasNext()) {
+XMLEvent event = eventReader.nextEvent();
+
+if (isGroupIdentifier(event)) {
+groupIdentifier =
getGroupIdentifier(eventReader.nextEvent());
+}
+
+if (isSensitiveElement(event)) {
+
xmlWriter.add(updateStartElementEncryptionAttribute(event));
+
xmlWriter.add(cryptoOperationOnCharacters(eventReader.nextEvent(),
groupIdentifier, getPropertyName(event)));
+} else {
+try {
+xmlWriter.add(event);
+} catch (Exception e) {
+throw new RuntimeException("Failed operation on XML
content", e);
+}
+}
+}
+
+eventReader.close();
+xmlWriter.flush();
+xmlWriter.close();
+} catch (Exception e) {
+throw new RuntimeException("Failed operation on XML content", e);
+}
+}
+
+/**
+ * Update the StartElement 'encryption' attribute for a sensitive value to
add or remove the respective encryption details eg. encryption="aes/gcm/128"
+ * @param xmlEvent A 'sensitive' StartElement that contains the
'encryption' tag attribute
+ * @return The updated StartElement
+ */
+protected abstract StartElement
updateStartElementEncryptionAttribute(final XMLEvent xmlEvent);
+
+/**
+ * Perform an encrypt or decrypt cryptographic operation on a Characters
element
+ * @param xmlEvent A Characters XmlEvent
+ * @param groupIdentifier The XML tag
+ * @return The Characters XmlEvent that has been updated by the
cryptographic operation
+ */
+protected abstract Characters cryptoOperationOnCharacters(final XMLEvent
xmlEvent, final String groupIdentifier, final String
[jira] [Updated] (NIFI-10389) Upgrade AWS Kinesis to 1.14.8
[ https://issues.apache.org/jira/browse/NIFI-10389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R updated NIFI-10389: -- Description: Upgrade AWS Kinesis to 1.14.8, from 1.14.7 to mitigate 2 CVEs: [CVE-2021-22569|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-22569] [CVE-2021-44832|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44832] was:Upgrade AWS Kinesis to 1.14.8, from 1.14.7 to mitigate a CVE > Upgrade AWS Kinesis to 1.14.8 > - > > Key: NIFI-10389 > URL: https://issues.apache.org/jira/browse/NIFI-10389 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade AWS Kinesis to 1.14.8, from 1.14.7 to mitigate 2 CVEs: > [CVE-2021-22569|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-22569] > [CVE-2021-44832|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44832] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 opened a new pull request, #6328: NIFI-10389 Upgrade AWS Kinesis to 1.14.8
mr1716 opened a new pull request, #6328: URL: https://github.com/apache/nifi/pull/6328 # Summary [NIFI-10389](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-10389) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10389) Upgrade AWS Kinesis to 1.14.8
Mike R created NIFI-10389: - Summary: Upgrade AWS Kinesis to 1.14.8 Key: NIFI-10389 URL: https://issues.apache.org/jira/browse/NIFI-10389 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.17.0 Reporter: Mike R Upgrade AWS Kinesis to 1.14.8, from 1.14.7 to mitigate a CVE -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-9559) Zookeeper Client Can't Reconnect - Session timeout has elapsed while SUSPENDED
[ https://issues.apache.org/jira/browse/NIFI-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583897#comment-17583897 ] Patrick Sodre commented on NIFI-9559: - We are also experiencing this issue when our ZooKeeper ensemble restarts. Our configuration is slightly different: ZooKeeper is 3.7.1 with a 3-node ensemble running on AWS ECS Fargate with DNS entries for the connection string. The IP addresses are dynamic in Fargate. We have multiple NiFi clusters, but I saw the issue happening in both 1.15.3 and 1.16.3. Our NiFi nodes are also running in AWS. I am very confident the SecurityGroups in AWS are properly configured and allow traffic between the NiFi clusters and the Zookeeper nodes. > Zookeeper Client Can't Reconnect - Session timeout has elapsed while SUSPENDED > -- > > Key: NIFI-9559 > URL: https://issues.apache.org/jira/browse/NIFI-9559 > Project: Apache NiFi > Issue Type: Bug >Reporter: Shawn Weeks >Assignee: Nathan Gough >Priority: Minor > Attachments: nifi_and_zookeeper_logs.txt, nifi_error.log > > > After a loss of connection to Zookeeper a NiFi node never successfully > reconnects to the Zookeeper or the Cluster and instead returns errors about > no Cluster Coordinator and a Session timeout has elapsed while SUSPENDED > repeatedly until you restart NiFi. > The error described is the same one at > https://issues.apache.org/jira/browse/CURATOR-405 however that patch has been > in NiFi for several versions now. > NiFi version is 1.15.3 and Zookeeper 3.6.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
turcsanyip commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r953145681
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.adapters;
+
+import com.hivemq.client.mqtt.datatypes.MqttQos;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5Connect;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5ConnectBuilder;
+import com.hivemq.client.mqtt.mqtt5.message.subscribe.suback.Mqtt5SubAck;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processors.mqtt.common.MqttCallback;
+import org.apache.nifi.processors.mqtt.common.MqttClient;
+import org.apache.nifi.processors.mqtt.common.MqttConnectionProperties;
+import org.apache.nifi.processors.mqtt.common.MqttException;
+import org.apache.nifi.processors.mqtt.common.ReceivedMqttMessage;
+import org.apache.nifi.processors.mqtt.common.StandardMqttMessage;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Objects;
+import java.util.concurrent.CompletableFuture;
+
+public class HiveMqV5ClientAdapter implements MqttClient {
+
+private final Mqtt5BlockingClient mqtt5BlockingClient;
+private final ComponentLog logger;
+
+private MqttCallback callback;
+
+public HiveMqV5ClientAdapter(Mqtt5BlockingClient mqtt5BlockingClient,
ComponentLog logger) {
+this.mqtt5BlockingClient = mqtt5BlockingClient;
+this.logger = logger;
+}
+
+@Override
+public boolean isConnected() {
+return mqtt5BlockingClient.getState().isConnected();
+}
+
+@Override
+public void connect(MqttConnectionProperties connectionProperties) {
+logger.debug("Connecting to broker");
+
+final Mqtt5ConnectBuilder connectBuilder = Mqtt5Connect.builder()
+.keepAlive(connectionProperties.getKeepAliveInterval());
+
+final boolean cleanSession = connectionProperties.isCleanSession();
+connectBuilder.cleanStart(cleanSession);
+if (!cleanSession) {
+
connectBuilder.sessionExpiryInterval(connectionProperties.getSessionExpiryInterval());
+}
+
+final String lastWillTopic = connectionProperties.getLastWillTopic();
+if (lastWillTopic != null) {
+connectBuilder.willPublish()
+.topic(lastWillTopic)
+
.payload(connectionProperties.getLastWillMessage().getBytes())
+.retain(connectionProperties.getLastWillRetain())
+
.qos(MqttQos.fromCode(connectionProperties.getLastWillQOS()))
+.applyWillPublish();
+}
+
+final String username = connectionProperties.getUsername();
+final String password = connectionProperties.getPassword();
+if (username != null && password != null) {
+connectBuilder.simpleAuth()
+.username(connectionProperties.getUsername())
+.password(password.getBytes(StandardCharsets.UTF_8))
+.applySimpleAuth();
+}
+
+final Mqtt5Connect mqtt5Connect = connectBuilder.build();
+mqtt5BlockingClient.connect(mqtt5Connect);
+}
+
+@Override
+public void disconnect(long disconnectTimeout) {
+logger.debug("Disconnecting client");
+// Currently it is not possible to set timeout for disconnect with
HiveMQ Client. (Only connect timeout exists.)
+mqtt5BlockingClient.disconnect();
+}
+
+@Override
+public void close() {
+// there is no paho's close equivalent in hivemq client
+}
+
+@Override
+public void publish(String topic, StandardMqttMessage message) {
+logger.debug("Publishing message to {} with QoS: {}", topic,
message.getQos());
+
+mqtt5BlockingClient.publishWith()
+.topic(topic)
+.payload(message.getPayload())
+.retain(message.isRetained())
+
[GitHub] [nifi] thenatog commented on a diff in pull request #6273: NIFI-9953 - The config encryption tool is too complicated to use and can be simplified
thenatog commented on code in PR #6273:
URL: https://github.com/apache/nifi/pull/6273#discussion_r953140358
##
nifi-toolkit/nifi-property-encryptor-tool/src/main/java/org/apache/nifi/xml/XmlCryptoParser.java:
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.xml;
+
+import org.apache.nifi.properties.SensitivePropertyProvider;
+import org.apache.nifi.properties.SensitivePropertyProviderFactory;
+import org.apache.nifi.properties.scheme.ProtectionScheme;
+
+import javax.xml.namespace.QName;
+import javax.xml.stream.XMLEventFactory;
+import javax.xml.stream.XMLEventReader;
+import javax.xml.stream.XMLEventWriter;
+import javax.xml.stream.XMLInputFactory;
+import javax.xml.stream.XMLOutputFactory;
+import javax.xml.stream.XMLStreamException;
+import javax.xml.stream.events.Attribute;
+import javax.xml.stream.events.Characters;
+import javax.xml.stream.events.StartElement;
+import javax.xml.stream.events.XMLEvent;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.Objects;
+
+public abstract class XmlCryptoParser {
+
+protected static final String ENCRYPTION_ATTRIBUTE_NAME = "encryption";
+protected static final String PARENT_IDENTIFIER = "identifier";
+protected static final String PROPERTY_ELEMENT = "property";
+
+protected final SensitivePropertyProvider cryptoProvider;
+protected SensitivePropertyProviderFactory providerFactory;
+
+public XmlCryptoParser(final SensitivePropertyProviderFactory
providerFactory, final ProtectionScheme scheme) {
+this.providerFactory = providerFactory;
+cryptoProvider = providerFactory.getProvider(scheme);
+}
+
+protected void cryptographicXmlOperation(final InputStream
encryptedXmlContent, final OutputStream decryptedOutputStream) {
+final XMLOutputFactory factory = XMLOutputFactory.newInstance();
+factory.setProperty("com.ctc.wstx.outputValidateStructure", false);
+
+try {
+final XMLEventReader eventReader =
getXMLReader(encryptedXmlContent);
+final XMLEventWriter xmlWriter =
factory.createXMLEventWriter(decryptedOutputStream);
+String groupIdentifier = "";
+
+while(eventReader.hasNext()) {
+XMLEvent event = eventReader.nextEvent();
+
+if (isGroupIdentifier(event)) {
+groupIdentifier =
getGroupIdentifier(eventReader.nextEvent());
+}
+
+if (isSensitiveElement(event)) {
+
xmlWriter.add(updateStartElementEncryptionAttribute(event));
+
xmlWriter.add(cryptoOperationOnCharacters(eventReader.nextEvent(),
groupIdentifier, getPropertyName(event)));
+} else {
+try {
+xmlWriter.add(event);
+} catch (Exception e) {
+throw new RuntimeException("Failed operation on XML
content", e);
+}
+}
+}
+
+eventReader.close();
+xmlWriter.flush();
+xmlWriter.close();
+} catch (Exception e) {
+throw new RuntimeException("Failed operation on XML content", e);
+}
+}
+
+/**
+ * Update the StartElement 'encryption' attribute for a sensitive value to
add or remove the respective encryption details eg. encryption="aes/gcm/128"
+ * @param xmlEvent A 'sensitive' StartElement that contains the
'encryption' tag attribute
+ * @return The updated StartElement
+ */
+protected abstract StartElement
updateStartElementEncryptionAttribute(final XMLEvent xmlEvent);
+
+/**
+ * Perform an encrypt or decrypt cryptographic operation on a Characters
element
+ * @param xmlEvent A Characters XmlEvent
+ * @param groupIdentifier The XML tag
+ * @return The Characters XmlEvent that has been updated by the
cryptographic operation
+ */
+protected abstract Characters cryptoOperationOnCharacters(final XMLEvent
xmlEvent, final String groupIdentifier, final String propertyName);
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
turcsanyip commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r953140061
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/AbstractMQTTProcessor.java:
##
@@ -289,88 +288,35 @@ public Collection customValidate(final
ValidationContext valid
return results;
}
-public static Properties transformSSLContextService(SSLContextService
sslContextService){
-Properties properties = new Properties();
-if (sslContextService.getSslAlgorithm() != null) {
-properties.setProperty("com.ibm.ssl.protocol",
sslContextService.getSslAlgorithm());
-}
-if (sslContextService.getKeyStoreFile() != null) {
-properties.setProperty("com.ibm.ssl.keyStore",
sslContextService.getKeyStoreFile());
-}
-if (sslContextService.getKeyStorePassword() != null) {
-properties.setProperty("com.ibm.ssl.keyStorePassword",
sslContextService.getKeyStorePassword());
-}
-if (sslContextService.getKeyStoreType() != null) {
-properties.setProperty("com.ibm.ssl.keyStoreType",
sslContextService.getKeyStoreType());
-}
-if (sslContextService.getTrustStoreFile() != null) {
-properties.setProperty("com.ibm.ssl.trustStore",
sslContextService.getTrustStoreFile());
-}
-if (sslContextService.getTrustStorePassword() != null) {
-properties.setProperty("com.ibm.ssl.trustStorePassword",
sslContextService.getTrustStorePassword());
-}
-if (sslContextService.getTrustStoreType() != null) {
-properties.setProperty("com.ibm.ssl.trustStoreType",
sslContextService.getTrustStoreType());
-}
-return properties;
-}
-
-protected void onScheduled(final ProcessContext context){
-broker =
context.getProperty(PROP_BROKER_URI).evaluateAttributeExpressions().getValue();
-brokerUri = broker.endsWith("/") ? broker : broker + "/";
-clientID =
context.getProperty(PROP_CLIENTID).evaluateAttributeExpressions().getValue();
-
-if (clientID == null) {
-clientID = UUID.randomUUID().toString();
-}
-
-connOpts = new MqttConnectOptions();
-
connOpts.setCleanSession(context.getProperty(PROP_CLEAN_SESSION).asBoolean());
-
connOpts.setKeepAliveInterval(context.getProperty(PROP_KEEP_ALIVE_INTERVAL).asInteger());
-
connOpts.setMqttVersion(context.getProperty(PROP_MQTT_VERSION).asInteger());
-
connOpts.setConnectionTimeout(context.getProperty(PROP_CONN_TIMEOUT).asInteger());
-
-PropertyValue sslProp = context.getProperty(PROP_SSL_CONTEXT_SERVICE);
-if (sslProp.isSet()) {
-Properties sslProps =
transformSSLContextService((SSLContextService) sslProp.asControllerService());
-connOpts.setSSLProperties(sslProps);
-}
-
-PropertyValue lastWillTopicProp =
context.getProperty(PROP_LAST_WILL_TOPIC);
-if (lastWillTopicProp.isSet()){
-String lastWillMessage =
context.getProperty(PROP_LAST_WILL_MESSAGE).getValue();
-PropertyValue lastWillRetain =
context.getProperty(PROP_LAST_WILL_RETAIN);
-Integer lastWillQOS =
context.getProperty(PROP_LAST_WILL_QOS).asInteger();
-connOpts.setWill(lastWillTopicProp.getValue(),
lastWillMessage.getBytes(), lastWillQOS, lastWillRetain.isSet() ?
lastWillRetain.asBoolean() : false);
-}
-
-
-PropertyValue usernameProp = context.getProperty(PROP_USERNAME);
-if(usernameProp.isSet()) {
-
connOpts.setUserName(usernameProp.evaluateAttributeExpressions().getValue());
-
connOpts.setPassword(context.getProperty(PROP_PASSWORD).getValue().toCharArray());
-}
+protected void onScheduled(final ProcessContext context) {
+clientProperties = getMqttClientProperties(context);
+connectionProperties = getMqttConnectionProperties(context);
}
protected void onStopped() {
-try {
-logger.info("Disconnecting client");
-mqttClient.disconnect(DISCONNECT_TIMEOUT);
-} catch(MqttException me) {
-logger.error("Error disconnecting MQTT client due to {}", new
Object[]{me.getMessage()}, me);
-}
+// Since client is created in the onTrigger method it can happen that
it never will be created because of an initialization error.
+// We are preventing additional nullPtrException here, but the clean
solution would be to create the client in the onScheduled method.
+if (mqttClient != null) {
+try {
+logger.info("Disconnecting client");
+mqttClient.disconnect(DISCONNECT_TIMEOUT);
+} catch (MqttException me) {
+logger.error("Error disconnecting MQTT client due to {}", new
Object[]{me.getMessage()}, me);
+}
+
+
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6301: NIFI-10356: Create GetHubSpot processor
exceptionfactory commented on code in PR #6301:
URL: https://github.com/apache/nifi/pull/6301#discussion_r953102705
##
nifi-nar-bundles/nifi-hubspot-bundle/nifi-hubspot-processors/src/main/java/org/apache/nifi/processors/hubspot/GetHubSpot.java:
##
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.hubspot;
+
+import com.fasterxml.jackson.core.JsonEncoding;
+import com.fasterxml.jackson.core.JsonFactory;
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.core.JsonToken;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.PrimaryNodeOnly;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.TriggerWhenEmpty;
+import org.apache.nifi.annotation.configuration.DefaultSettings;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.web.client.api.HttpResponseEntity;
+import org.apache.nifi.web.client.api.HttpResponseStatus;
+import org.apache.nifi.web.client.api.HttpUriBuilder;
+import org.apache.nifi.web.client.provider.api.WebClientServiceProvider;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+
+@PrimaryNodeOnly
+@TriggerSerially
+@TriggerWhenEmpty
+@InputRequirement(Requirement.INPUT_FORBIDDEN)
+@Tags({"hubspot"})
+@CapabilityDescription("Retrieves JSON data from a private HubSpot
application."
++ " Supports incremental retrieval: Users can set the \"limit\"
property which serves as the upper limit of the retrieved objects."
++ " When this property is set the processor will retrieve new records.
This processor is intended to be run on the Primary Node only.")
+@Stateful(scopes = Scope.CLUSTER, description = "When the 'Limit' attribute is
set, the paging cursor is saved after executing a request."
++ " Only the objects after the paging cursor will be retrieved. The
maximum number of retrieved objects is the 'Limit' attribute."
++ " State is stored across the cluster so that this Processor can be
run on Primary Node only and if a new Primary Node is selected,"
++ " the new node can pick up where the previous node left off, without
duplicating the data.")
+@DefaultSettings(yieldDuration = "10 sec")
+public class GetHubSpot extends AbstractProcessor {
+
+static final PropertyDescriptor CRM_ENDPOINT = new
PropertyDescriptor.Builder()
+.name("crm-endpoint")
+.displayName("HubSpot CRM API Endpoint")
+.description("The HubSpot CRM API endpoint to which the Processor
will send requests")
+.required(true)
+.allowableValues(CrmEndpoint.class)
+.build();
+
+static final PropertyDescriptor ACCESS_TOKEN = new
PropertyDescriptor.Builder()
+.name("access-token")
+.displayName("Access Token")
+.description("Access Token to authenticate requests")
+
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6301: NIFI-10356: Create GetHubSpot processor
exceptionfactory commented on code in PR #6301: URL: https://github.com/apache/nifi/pull/6301#discussion_r953100112 ## nifi-nar-bundles/nifi-hubspot-bundle/nifi-hubspot-processors/src/main/java/org/apache/nifi/processors/hubspot/GetHubSpot.java: ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.hubspot; + +import com.fasterxml.jackson.core.JsonEncoding; +import com.fasterxml.jackson.core.JsonFactory; +import com.fasterxml.jackson.core.JsonGenerator; +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.JsonToken; +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; +import org.apache.nifi.annotation.behavior.PrimaryNodeOnly; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.TriggerWhenEmpty; +import org.apache.nifi.annotation.configuration.DefaultSettings; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.expression.ExpressionLanguageScope; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.web.client.api.HttpResponseEntity; +import org.apache.nifi.web.client.api.HttpUriBuilder; +import org.apache.nifi.web.client.provider.api.WebClientServiceProvider; + +import java.io.IOException; +import java.net.URI; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.atomic.AtomicInteger; + +@PrimaryNodeOnly +@TriggerSerially Review Comment: Thanks for the reply @Lehel44, these annotations make sense given the approach implemented in other source processors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10321) Improve Expired JWT Error Message Wording
[
https://issues.apache.org/jira/browse/NIFI-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10321:
Summary: Improve Expired JWT Error Message Wording (was: invalid_token
error after SAML session timeout)
> Improve Expired JWT Error Message Wording
> -
>
> Key: NIFI-10321
> URL: https://issues.apache.org/jira/browse/NIFI-10321
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core UI
>Affects Versions: 1.17.0
> Environment: CentOS 8, NIFI 1.17.0, Keycloak 19.0.1
>Reporter: macdoor615
>Assignee: David Handermann
>Priority: Minor
> Attachments: 截屏2022-08-05 13.31.47.png
>
>
> I follow
> [https://bryanbende.com/development/2021/02/17/apache-nifi-saml-keycloak] to
> config nifi 1.17.0. NIFI can login successful with SAML Authentication with
> Keycloak 19.0.1. But when nifi times out with SAML session. NIFI UI gives the
> following error.
> {code:java}
> Unauthorized error="invalid_token", error_description="An error occurred
> while attempting to decode the Jwt: Expired JWT",
> error_uri="https://tools.ietf.org/html/rfc6750#section-3.1"{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] markap14 commented on pull request #6131: NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key
markap14 commented on PR #6131:
URL: https://github.com/apache/nifi/pull/6131#issuecomment-1224840438
@greyp9 So, given the above (i will address in reverse order because I think
you started with the hardest to describe and got easier as you went down the
list) :) ...
The last point, about "Message Key Field" property I think is perfectly
accurate.
As for the headers:
- I think what you said is accurate, but to clarify:
- If (Use Wrapper) - the headers to send would be a single header. Its name
would be "headerA" and its value would be "headerAValue". FlowFile attributes
would not be sent as headers.
- Else, the headers would be any FlowFile attribute that matches the
"Attributes to Send as Headers (Regex)" property
Now, as for the other...
```
if (Use Wrapper) {
Kafka Record:
Key = { "type": "person" }
Value = { "name": "Mark", "number": 49 }
Headers = A single header with name "headerA", value "headerAValue"
} else {
Kafka Record:
Key =
Value =
{
"key": {
"type": "person"
},
"value": {
"name": "Mark",
"number": 49
},
"headers": {
"headerA": "headerAValue"
}
}
Headers =
}
```
So, in short, if Use Wrapper, the incoming FlowFile must have 3 fields:
Key. This becomes the kafka message key.
Value. This becomes the contents of the kafka message.
Headers. This becomes the headers attached to the kafka message.
Any other fields, such as metadata, would be ignored.
If NOT using wrapper, it would function as it always has. The entire
contents of the Record go as the kafka message payload. The key and headers are
determined based on the configured "Message Key Field" and "Attributes to Send
as Headers (Regex)" properties.
Does that make sense? Or have I confused things even worse? :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-10388) Upgrade xmlunit-matchers to 2.9.0 to eliminate CVEs
[ https://issues.apache.org/jira/browse/NIFI-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10388. - Fix Version/s: 1.18.0 Assignee: Mike R Resolution: Fixed > Upgrade xmlunit-matchers to 2.9.0 to eliminate CVEs > --- > > Key: NIFI-10388 > URL: https://issues.apache.org/jira/browse/NIFI-10388 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Fix For: 1.18.0 > > Time Spent: 20m > Remaining Estimate: 0h > > xmlunit-matchers to 2.9.0, from the current 2.6.3 to eliminate vulnerability > from CVE > [CVE-2020-15250|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15250] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10388) Upgrade xmlunit-matchers to 2.9.0 to eliminate CVEs
[ https://issues.apache.org/jira/browse/NIFI-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583836#comment-17583836 ] ASF subversion and git services commented on NIFI-10388: Commit 4a2c0beb7f2b3e67889cf74b4a3ff94ee736209f in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4a2c0beb7f ] NIFI-10388 Upgraded xmlunit-matchers from 2.6.3 to 2.9.0 This closes #6326 Signed-off-by: David Handermann > Upgrade xmlunit-matchers to 2.9.0 to eliminate CVEs > --- > > Key: NIFI-10388 > URL: https://issues.apache.org/jira/browse/NIFI-10388 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > xmlunit-matchers to 2.9.0, from the current 2.6.3 to eliminate vulnerability > from CVE > [CVE-2020-15250|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15250] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6326: NIFI-10388 Upgrade xmlunit-matchers to 2.9.0
exceptionfactory closed pull request #6326: NIFI-10388 Upgrade xmlunit-matchers to 2.9.0 URL: https://github.com/apache/nifi/pull/6326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10335) Upgrade Azure-Core to 1.31.0
[ https://issues.apache.org/jira/browse/NIFI-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10335: Priority: Minor (was: Major) > Upgrade Azure-Core to 1.31.0 > > > Key: NIFI-10335 > URL: https://issues.apache.org/jira/browse/NIFI-10335 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike R >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade Azure-Core to 1.31.0 from 1.26.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10335) Upgrade Azure-Core to 1.31.0
[ https://issues.apache.org/jira/browse/NIFI-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10335: Issue Type: Improvement (was: Bug) > Upgrade Azure-Core to 1.31.0 > > > Key: NIFI-10335 > URL: https://issues.apache.org/jira/browse/NIFI-10335 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade Azure-Core to 1.31.0 from 1.26.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on pull request #6131: NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key
greyp9 commented on PR #6131:
URL: https://github.com/apache/nifi/pull/6131#issuecomment-1224730836
> ```
> {
> "key": {
> "type": "person"
> },
> "value": {
> "name": "Mark",
> "number": 49
> },
> "headers": {
> "headerA": "headerAValue"
> }
> }
> ```
Thanks. There are enough permutations related to these changes such that
I'm not sure what should be emitted for all of them. I've made some
simplifying assumptions; those may need adjustments.
Maybe a concrete example would help. Given the record above, and the
processor PublishKafkaRecord_2_6:
[Processor Property] PublishStrategy
```
if (Use Wrapper) {
Kafka Record:
Key = ?
Value = ?
Headers = ?
} else {
Kafka Record:
Key = ?
Value = ?
Headers = ?
}
```
---
[Processor Property] Attributes To Send as Headers (Regex)
```
if (Use Wrapper) {
send attributes in wrapper record (do not add to kafka headers)
} else {
send attributes in kafka headers
}
```
is this high-level logic valid?
---
[Processor Property] Message Key Field
```
if (Use Wrapper) {
ignore (null key)
} else {
send key (if record field exists) in Kafka key
}
```
is this right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1385: MINIFICPP-1901 Add agent identity to Prometheus metrics
lordgamez commented on code in PR #1385: URL: https://github.com/apache/nifi-minifi-cpp/pull/1385#discussion_r953021413 ## METRICS.md: ## @@ -46,77 +46,88 @@ To use the publisher a port should also be configured where the metrics will be nifi.metrics.publisher.PrometheusMetricsPublisher.port=9936 -The last option defines which metric classes should be exposed through the metrics publisher in configured with a comma separated value: +The following option defines which metric classes should be exposed through the metrics publisher in configured with a comma separated value: # in minifi.properties nifi.metrics.publisher.metrics=QueueMetrics,RepositoryMetrics,GetFileMetrics,DeviceInfoNode,FlowInformation +An agent identifier should also be defined to identify which agent the metric is exposed from. If not set the hostname is used as the identifier. Review Comment: Updated in 04aa1120bda6a70d66461a87cddffc095c641499 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1906) ListenHTTPTests runs too long
[ https://issues.apache.org/jira/browse/MINIFICPP-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi resolved MINIFICPP-1906. -- Fix Version/s: 0.13.0 Resolution: Fixed > ListenHTTPTests runs too long > - > > Key: MINIFICPP-1906 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1906 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ádám Markovics >Assignee: Gábor Gyimesi >Priority: Minor > Fix For: 0.13.0 > > Time Spent: 20m > Remaining Estimate: 0h > > It used to run for like 10-15 seconds, now a typical run is 1 min 45 secs. > This comes from CivetWeb version update in > https://github.com/apache/nifi-minifi-cpp/commit/0bf8ce968bdebfa6867eb339049bfabca19e5df0. > If lib version is reverted, the issue disappears. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
szaszm commented on PR #1383: URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#issuecomment-1224655249 pushed a rebase to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #1367: MINIFICPP-1822 - Add alert capability
szaszm commented on PR #1367: URL: https://github.com/apache/nifi-minifi-cpp/pull/1367#issuecomment-1224653894 pushed a rebase to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1401: MINIFICPP-1918 - Fix windows build
szaszm closed pull request #1401: MINIFICPP-1918 - Fix windows build URL: https://github.com/apache/nifi-minifi-cpp/pull/1401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1403: MINIFICPP-1906 Restore original timeout for CivetServer
szaszm closed pull request #1403: MINIFICPP-1906 Restore original timeout for CivetServer URL: https://github.com/apache/nifi-minifi-cpp/pull/1403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1402: MINIFICPP-1919 centos HTTPClientTests fix
szaszm closed pull request #1402: MINIFICPP-1919 centos HTTPClientTests fix URL: https://github.com/apache/nifi-minifi-cpp/pull/1402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on pull request #6131: NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key
greyp9 commented on PR #6131: URL: https://github.com/apache/nifi/pull/6131#issuecomment-1224612052 > Thanks for updating @greyp9 . Trying this out again. I just tried sending the following content via PublishKafkaRecord_2_6: > I set Publish Strategy to "Use Wrapper" and used a JsonTreeReader as the record reader. But I encountered a NullPointerException: > Updated to validate content of message key. > One other thing that I noticed, though it's minor: When I change the "Publish Strategy" to "Use Wrapper" I'm given the option to configure the "Record Key Writer" property. But this is way down the list, about 8 or 10 properties down, so it's not at all obvious that it's made available. Probably want to move the "Record Key Writer" just after the "Publish Strategy" property sends it depends on it. Moved to just after "Publish Strategy". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] tpalfy opened a new pull request, #6327: NIFI-10379 - FetchGoogleDrive improvements and fixes
tpalfy opened a new pull request, #6327: URL: https://github.com/apache/nifi/pull/6327 Fix: Don't send anything to 'success' relationship when a general error occurs after some records have successfully processed. More proper handling of returned values when calling methods on the ProcessSession. Record-based processing requires predefined schem for the input. Added 'addtionalDetails' accordingly. # Summary [NIFI-10379](https://issues.apache.org/jira/browse/NIFI-10379) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10379) FechGoogleDrive record-based input handling improvement
[ https://issues.apache.org/jira/browse/NIFI-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Palfy updated NIFI-10379: --- Description: *FechGoogleDrive* can read input from flowfile contents as records. However the following cases are not handled correctly: # There is no property on the processor to configure what field in the records are to be used as file identifiers (based on which fetch is executed). Instead it is a hardcoded string value. *Edit:* Upon further consideration the record-based approach poses technical challenges if we wanted to be properly customizable. Instead we will consider this to mainly support the output of the ListGoogleDrive processor. Still can be used from differenc sources but the expected schema is pre-defined. # It is possible for the processor to fetch successfully some files but later experience a general error which sends the incoming flowfile to an error relationship. This is a mixed result (some records succeed but then _all records_ fail) and should not happen. Either all records should be processed (resulting one flowfile for each sent to a success or an error relationship) or none of them (resulting only the incoming flowfile being sent to another error relationship). # _session.xxx(flowFile)_ calls are used as if those were _viod_ (and operated on the incoming flowFile). Should use the returned flowfile instead. was: *FechGoogleDrive* can read input from flowfile contents as records. However the following cases are not handled correctly: # There is no property on the processor to configure what field in the records are to be used as file identifiers (based on which fetch is executed). Instead it is a hardcoded string value. # It is possible for the processor to fetch successfully some files but later experience a general error which sends the incoming flowfile to an error relationship. This is a mixed result (some records succeed but then _all records_ fail) and should not happen. Either all records should be processed (resulting one flowfile for each sent to a success or an error relationship) or none of them (resulting only the incoming flowfile being sent to another error relationship). # _session.xxx(flowFile)_ calls are used as if those were _viod_ (and operated on the incoming flowFile). Should use the returned flowfile instead. > FechGoogleDrive record-based input handling improvement > --- > > Key: NIFI-10379 > URL: https://issues.apache.org/jira/browse/NIFI-10379 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > > *FechGoogleDrive* can read input from flowfile contents as records. > However the following cases are not handled correctly: > # There is no property on the processor to configure what field in the > records are to be used as file identifiers (based on which fetch is > executed). Instead it is a hardcoded string value. > *Edit:* Upon further consideration the record-based approach poses technical > challenges if we wanted to be properly customizable. > Instead we will consider this to mainly support the output of the > ListGoogleDrive processor. Still can be used from differenc sources but the > expected schema is pre-defined. > # It is possible for the processor to fetch successfully some files but > later experience a general error which sends the incoming flowfile to an > error relationship. This is a mixed result (some records succeed but then > _all records_ fail) and should not happen. Either all records should be > processed (resulting one flowfile for each sent to a success or an error > relationship) or none of them (resulting only the incoming flowfile being > sent to another error relationship). > # _session.xxx(flowFile)_ calls are used as if those were _viod_ (and > operated on the incoming flowFile). Should use the returned flowfile instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10387) Scripted components incorrectly marking whether a script needs to be reloaded
[ https://issues.apache.org/jira/browse/NIFI-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-10387: -- Fix Version/s: 1.18.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Scripted components incorrectly marking whether a script needs to be reloaded > - > > Key: NIFI-10387 > URL: https://issues.apache.org/jira/browse/NIFI-10387 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.18.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In many of the scripted components (InvokeScriptedProcessor, scripted > controller services, etc.) there is a negative logic bug when setting the > scriptNeedsReload variable to indicate that the script should be reloaded. > Specifically, if the script does not load successfully, scriptNeedsReload is > set false when it should be true. In most/all cases, this error does not > manifest because there is also a check to make sure there is an actual > lookupService object retrieved from the (successfully loaded) script. However > there may be situations in which this negative logic bug could manifest and > thus it should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10387) Scripted components incorrectly marking whether a script needs to be reloaded
[ https://issues.apache.org/jira/browse/NIFI-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583777#comment-17583777 ] ASF subversion and git services commented on NIFI-10387: Commit 6f0ca8730411d01e5c5c2f845a5387020aee60ba in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6f0ca87304 ] NIFI-10387: Fixed negative logic bug in scripted components (#6325) > Scripted components incorrectly marking whether a script needs to be reloaded > - > > Key: NIFI-10387 > URL: https://issues.apache.org/jira/browse/NIFI-10387 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > In many of the scripted components (InvokeScriptedProcessor, scripted > controller services, etc.) there is a negative logic bug when setting the > scriptNeedsReload variable to indicate that the script should be reloaded. > Specifically, if the script does not load successfully, scriptNeedsReload is > set false when it should be true. In most/all cases, this error does not > manifest because there is also a check to make sure there is an actual > lookupService object retrieved from the (successfully loaded) script. However > there may be situations in which this negative logic bug could manifest and > thus it should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 merged pull request #6325: NIFI-10387: Fixed negative logic bug in scripted components
markap14 merged PR #6325: URL: https://github.com/apache/nifi/pull/6325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #6325: NIFI-10387: Fixed negative logic bug in scripted components
markap14 commented on PR #6325: URL: https://github.com/apache/nifi/pull/6325#issuecomment-1224467488 Thanks for the fix @mattyb149. Code changes look good, +1 will merge to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-10378) Toolkit: Add *.security.user.oidc.client.secret in default encryption
[ https://issues.apache.org/jira/browse/NIFI-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-10378: --- Assignee: Nathan Gough > Toolkit: Add *.security.user.oidc.client.secret in default encryption > - > > Key: NIFI-10378 > URL: https://issues.apache.org/jira/browse/NIFI-10378 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Navneet Kaur >Assignee: Nathan Gough >Priority: Minor > > Hi, > The below values are not encrypted by default through the toolkit: > 1. nifi.registry.security.user.oidc.client.secret > 2. nifi.security.user.oidc.client.secret > Requesting to add these encryption for default than being specified in > "*.sensitive.props.additional.keys". > Thanks, > Navneet -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-5232) HttpConnectionService controller service
[
https://issues.apache.org/jira/browse/NIFI-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583771#comment-17583771
]
David Handermann commented on NIFI-5232:
Thanks for following up on this [~mike.thomsen]! The {{nifi-scripting-nar}}
depends on {{{}nifi-standard-services-api-nar{}}}, so scripted components
should be able to leverage the new service with the
{{WebClientServiceProvider}} interface bundled as a standard Controller Service.
> HttpConnectionService controller service
>
>
> Key: NIFI-5232
> URL: https://issues.apache.org/jira/browse/NIFI-5232
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Mike Thomsen
>Priority: Major
>
> The functionality of InvokeHttp and related processors should be copied over
> to a controller service that can do much the same thing. This controller
> service would be able to handle all of the common scenarios with HTTP
> connections from processors going forward.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] nandorsoma commented on pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on PR #6225: URL: https://github.com/apache/nifi/pull/6225#issuecomment-1224352753 Thank you for the review @turcsanyip, @tpalfy and @exceptionfactory! I've tried to address your comments! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952894224
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.adapters;
+
+import com.hivemq.client.mqtt.datatypes.MqttQos;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5Client;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5Connect;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5ConnectBuilder;
+import org.apache.nifi.processors.mqtt.common.MqttConnectionProperties;
+import org.apache.nifi.processors.mqtt.common.NifiMqttCallback;
+import org.apache.nifi.processors.mqtt.common.NifiMqttClient;
+import org.apache.nifi.processors.mqtt.common.NifiMqttException;
+import org.apache.nifi.processors.mqtt.common.NifiMqttMessage;
+
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.TrustManagerFactory;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.StandardCharsets;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.cert.CertificateException;
+import java.util.Arrays;
+import java.util.Objects;
+
+public class HiveMqV5ClientAdapter implements NifiMqttClient {
+
+private final Mqtt5Client mqtt5Client;
+
+private NifiMqttCallback callback;
+
+public HiveMqV5ClientAdapter(Mqtt5BlockingClient mqtt5BlockingClient) {
+this.mqtt5Client = mqtt5BlockingClient;
+}
+
+@Override
+public boolean isConnected() {
+return mqtt5Client.getState().isConnected();
+}
+
+@Override
+public void connect(MqttConnectionProperties connectionProperties) throws
NifiMqttException {
+final Mqtt5ConnectBuilder connectBuilder = Mqtt5Connect.builder()
+.keepAlive(connectionProperties.getKeepAliveInterval());
+
+final boolean cleanSession = connectionProperties.isCleanSession();
+connectBuilder.cleanStart(cleanSession);
+if (!cleanSession) {
+
connectBuilder.sessionExpiryInterval(connectionProperties.getSessionExpiryInterval());
+}
+
+final String lastWillTopic = connectionProperties.getLastWillTopic();
+if (lastWillTopic != null) {
+connectBuilder.willPublish()
+.topic(lastWillTopic)
+
.payload(connectionProperties.getLastWillMessage().getBytes())
+.retain(connectionProperties.getLastWillRetain())
+
.qos(MqttQos.fromCode(connectionProperties.getLastWillQOS()))
+.applyWillPublish();
+}
+
+// checking for presence of password because username can be null
+final char[] password = connectionProperties.getPassword();
+if (password != null) {
+connectBuilder.simpleAuth()
+.username(connectionProperties.getUsername())
+.password(toBytes(password))
+.applySimpleAuth();
+
+clearSensitive(connectionProperties.getPassword());
+clearSensitive(password);
+}
+
+final Mqtt5Connect mqtt5Connect = connectBuilder.build();
+mqtt5Client.toBlocking().connect(mqtt5Connect);
Review Comment:
Changed.
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952894025
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.adapters;
+
+import com.hivemq.client.mqtt.datatypes.MqttQos;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5Client;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5Connect;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5ConnectBuilder;
+import org.apache.nifi.processors.mqtt.common.MqttConnectionProperties;
+import org.apache.nifi.processors.mqtt.common.NifiMqttCallback;
+import org.apache.nifi.processors.mqtt.common.NifiMqttClient;
+import org.apache.nifi.processors.mqtt.common.NifiMqttException;
+import org.apache.nifi.processors.mqtt.common.NifiMqttMessage;
+
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.TrustManagerFactory;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.StandardCharsets;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.cert.CertificateException;
+import java.util.Arrays;
+import java.util.Objects;
+
+public class HiveMqV5ClientAdapter implements NifiMqttClient {
+
+private final Mqtt5Client mqtt5Client;
+
+private NifiMqttCallback callback;
+
+public HiveMqV5ClientAdapter(Mqtt5BlockingClient mqtt5BlockingClient) {
+this.mqtt5Client = mqtt5BlockingClient;
+}
+
+@Override
+public boolean isConnected() {
+return mqtt5Client.getState().isConnected();
+}
+
+@Override
+public void connect(MqttConnectionProperties connectionProperties) throws
NifiMqttException {
+final Mqtt5ConnectBuilder connectBuilder = Mqtt5Connect.builder()
+.keepAlive(connectionProperties.getKeepAliveInterval());
+
+final boolean cleanSession = connectionProperties.isCleanSession();
+connectBuilder.cleanStart(cleanSession);
+if (!cleanSession) {
+
connectBuilder.sessionExpiryInterval(connectionProperties.getSessionExpiryInterval());
+}
+
+final String lastWillTopic = connectionProperties.getLastWillTopic();
+if (lastWillTopic != null) {
+connectBuilder.willPublish()
+.topic(lastWillTopic)
+
.payload(connectionProperties.getLastWillMessage().getBytes())
+.retain(connectionProperties.getLastWillRetain())
+
.qos(MqttQos.fromCode(connectionProperties.getLastWillQOS()))
+.applyWillPublish();
+}
+
+// checking for presence of password because username can be null
+final char[] password = connectionProperties.getPassword();
+if (password != null) {
+connectBuilder.simpleAuth()
+.username(connectionProperties.getUsername())
+.password(toBytes(password))
+.applySimpleAuth();
+
+clearSensitive(connectionProperties.getPassword());
+clearSensitive(password);
+}
+
+final Mqtt5Connect mqtt5Connect = connectBuilder.build();
+mqtt5Client.toBlocking().connect(mqtt5Connect);
+}
+
+@Override
+public void disconnect(long disconnectTimeout) throws NifiMqttException {
+// Currently it is not possible to set timeout for disconnect with
HiveMQ Client. (Only connect timeout exists.)
+mqtt5Client.toBlocking().disconnect();
+}
+
+@Override
+public void close() throws NifiMqttException {
+// there is no paho's close equivalent in hivemq client
+}
+
+@Override
+public void publish(String topic, NifiMqttMessage message) throws
NifiMqttException {
+mqtt5Client.toAsync().publishWith()
+.topic(topic)
+
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r95265
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.adapters;
+
+import com.hivemq.client.mqtt.datatypes.MqttQos;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5Client;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5Connect;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5ConnectBuilder;
+import org.apache.nifi.processors.mqtt.common.MqttConnectionProperties;
+import org.apache.nifi.processors.mqtt.common.NifiMqttCallback;
+import org.apache.nifi.processors.mqtt.common.NifiMqttClient;
+import org.apache.nifi.processors.mqtt.common.NifiMqttException;
+import org.apache.nifi.processors.mqtt.common.NifiMqttMessage;
+
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.TrustManagerFactory;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.StandardCharsets;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.cert.CertificateException;
+import java.util.Arrays;
+import java.util.Objects;
+
+public class HiveMqV5ClientAdapter implements NifiMqttClient {
+
+private final Mqtt5Client mqtt5Client;
+
+private NifiMqttCallback callback;
+
+public HiveMqV5ClientAdapter(Mqtt5BlockingClient mqtt5BlockingClient) {
+this.mqtt5Client = mqtt5BlockingClient;
+}
+
+@Override
+public boolean isConnected() {
+return mqtt5Client.getState().isConnected();
+}
+
+@Override
+public void connect(MqttConnectionProperties connectionProperties) throws
NifiMqttException {
+final Mqtt5ConnectBuilder connectBuilder = Mqtt5Connect.builder()
+.keepAlive(connectionProperties.getKeepAliveInterval());
+
+final boolean cleanSession = connectionProperties.isCleanSession();
+connectBuilder.cleanStart(cleanSession);
+if (!cleanSession) {
+
connectBuilder.sessionExpiryInterval(connectionProperties.getSessionExpiryInterval());
+}
+
+final String lastWillTopic = connectionProperties.getLastWillTopic();
+if (lastWillTopic != null) {
+connectBuilder.willPublish()
+.topic(lastWillTopic)
+
.payload(connectionProperties.getLastWillMessage().getBytes())
+.retain(connectionProperties.getLastWillRetain())
+
.qos(MqttQos.fromCode(connectionProperties.getLastWillQOS()))
+.applyWillPublish();
+}
+
+// checking for presence of password because username can be null
+final char[] password = connectionProperties.getPassword();
+if (password != null) {
+connectBuilder.simpleAuth()
+.username(connectionProperties.getUsername())
+.password(toBytes(password))
+.applySimpleAuth();
Review Comment:
Mqtt v5 allows to use just password without username:
https://www.hivemq.com/blog/mqtt5-essentials-part2-foundational-changes-in-the-protocol/
(see Using passwords without usernames)
But as we agreed, now I'm adding username to the check and when needed we
will create a separate field on the processor to allow authentication just with
password.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r95265
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.adapters;
+
+import com.hivemq.client.mqtt.datatypes.MqttQos;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5Client;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5Connect;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5ConnectBuilder;
+import org.apache.nifi.processors.mqtt.common.MqttConnectionProperties;
+import org.apache.nifi.processors.mqtt.common.NifiMqttCallback;
+import org.apache.nifi.processors.mqtt.common.NifiMqttClient;
+import org.apache.nifi.processors.mqtt.common.NifiMqttException;
+import org.apache.nifi.processors.mqtt.common.NifiMqttMessage;
+
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.TrustManagerFactory;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.StandardCharsets;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.cert.CertificateException;
+import java.util.Arrays;
+import java.util.Objects;
+
+public class HiveMqV5ClientAdapter implements NifiMqttClient {
+
+private final Mqtt5Client mqtt5Client;
+
+private NifiMqttCallback callback;
+
+public HiveMqV5ClientAdapter(Mqtt5BlockingClient mqtt5BlockingClient) {
+this.mqtt5Client = mqtt5BlockingClient;
+}
+
+@Override
+public boolean isConnected() {
+return mqtt5Client.getState().isConnected();
+}
+
+@Override
+public void connect(MqttConnectionProperties connectionProperties) throws
NifiMqttException {
+final Mqtt5ConnectBuilder connectBuilder = Mqtt5Connect.builder()
+.keepAlive(connectionProperties.getKeepAliveInterval());
+
+final boolean cleanSession = connectionProperties.isCleanSession();
+connectBuilder.cleanStart(cleanSession);
+if (!cleanSession) {
+
connectBuilder.sessionExpiryInterval(connectionProperties.getSessionExpiryInterval());
+}
+
+final String lastWillTopic = connectionProperties.getLastWillTopic();
+if (lastWillTopic != null) {
+connectBuilder.willPublish()
+.topic(lastWillTopic)
+
.payload(connectionProperties.getLastWillMessage().getBytes())
+.retain(connectionProperties.getLastWillRetain())
+
.qos(MqttQos.fromCode(connectionProperties.getLastWillQOS()))
+.applyWillPublish();
+}
+
+// checking for presence of password because username can be null
+final char[] password = connectionProperties.getPassword();
+if (password != null) {
+connectBuilder.simpleAuth()
+.username(connectionProperties.getUsername())
+.password(toBytes(password))
+.applySimpleAuth();
Review Comment:
Mqtt v5 allows to use just password without username:
https://www.hivemq.com/blog/mqtt5-essentials-part2-foundational-changes-in-the-protocol/
(see Using passwords without usernames)
But as we agreed, now I'll add username to the check and when needed we will
create a separate field on the processor to allow authentication just with
password.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952884772
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/NifiMqttMessage.java:
##
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.common;
+
+public class NifiMqttMessage {
Review Comment:
At start I wanted to do as minimal change as I can, that's why I left it in
the original way. Nevertheless, you are right, I changed it.
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/MqttClientProperties.java:
##
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.common;
+
+import java.net.URI;
+
+public class MqttClientProperties {
+private URI brokerURI;
+private String clientID;
Review Comment:
Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952882215
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/MqttConstants.java:
##
@@ -67,14 +69,45 @@ public class MqttConstants {
*/
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_AUTO =
new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_DEFAULT),
-"AUTO",
+"v3 AUTO",
"Start with v3.1.1 and fallback to v3.1.0 if not supported
by a broker");
+public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_500 =
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_5_0.getNumericValue()),
+"v5.0");
+
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_311 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1_1.getNumericValue()),
"v3.1.1");
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_310 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1.getNumericValue()),
"v3.1.0");
+
+public enum MqttVersion {
+MQTT_VERSION_3_1(3),
+MQTT_VERSION_3_1_1(4),
+MQTT_VERSION_5_0(5);
+
+private final int numericValue;
+
+MqttVersion(int numericValue) {
+this.numericValue = numericValue;
+}
+
+public int getNumericValue() {
+return numericValue;
+}
+}
+
+public enum SupportedSchemes {
Review Comment:
Changed.
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/MqttClientFactory.java:
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.common;
+
+import com.hivemq.client.mqtt.mqtt5.Mqtt5Client;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5ClientBuilder;
+import org.apache.nifi.processors.mqtt.adapters.HiveMqV5ClientAdapter;
+import org.apache.nifi.processors.mqtt.adapters.PahoMqttClientAdapter;
+import org.eclipse.paho.client.mqttv3.MqttClient;
+import org.eclipse.paho.client.mqttv3.MqttException;
+import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence;
+
+import static
org.apache.nifi.processors.mqtt.common.MqttConstants.SupportedSchemes.SSL;
+import static
org.apache.nifi.processors.mqtt.common.MqttConstants.SupportedSchemes.WS;
+import static
org.apache.nifi.processors.mqtt.common.MqttConstants.SupportedSchemes.WSS;
+
+public class MqttClientFactory {
+public NifiMqttClient create(MqttClientProperties clientProperties,
MqttConnectionProperties connectionProperties) {
+switch (clientProperties.getMqttVersion()) {
+case 0:
+case 3:
+case 4:
+return createPahoMqttV3ClientAdapter(clientProperties);
+case 5:
+return createHiveMqV5ClientAdapter(clientProperties,
connectionProperties);
Review Comment:
Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952881717
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/MqttConstants.java:
##
@@ -67,14 +69,45 @@ public class MqttConstants {
*/
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_AUTO =
new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_DEFAULT),
-"AUTO",
+"v3 AUTO",
"Start with v3.1.1 and fallback to v3.1.0 if not supported
by a broker");
+public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_500 =
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_5_0.getNumericValue()),
+"v5.0");
+
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_311 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1_1.getNumericValue()),
"v3.1.1");
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_310 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1.getNumericValue()),
"v3.1.0");
+
+public enum MqttVersion {
+MQTT_VERSION_3_1(3),
+MQTT_VERSION_3_1_1(4),
+MQTT_VERSION_5_0(5);
Review Comment:
Changed.
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/MqttConstants.java:
##
@@ -67,14 +69,45 @@ public class MqttConstants {
*/
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_AUTO =
new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_DEFAULT),
-"AUTO",
+"v3 AUTO",
"Start with v3.1.1 and fallback to v3.1.0 if not supported
by a broker");
+public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_500 =
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_5_0.getNumericValue()),
+"v5.0");
+
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_311 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1_1.getNumericValue()),
"v3.1.1");
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_310 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1.getNumericValue()),
"v3.1.0");
+
+public enum MqttVersion {
+MQTT_VERSION_3_1(3),
+MQTT_VERSION_3_1_1(4),
+MQTT_VERSION_5_0(5);
+
+private final int numericValue;
Review Comment:
Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952881239
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/MqttConstants.java:
##
@@ -67,14 +69,45 @@ public class MqttConstants {
*/
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_AUTO =
new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_DEFAULT),
-"AUTO",
+"v3 AUTO",
"Start with v3.1.1 and fallback to v3.1.0 if not supported
by a broker");
+public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_500 =
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_5_0.getNumericValue()),
+"v5.0");
+
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_311 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1_1.getNumericValue()),
"v3.1.1");
public static final AllowableValue ALLOWABLE_VALUE_MQTT_VERSION_310 =
-new
AllowableValue(String.valueOf(MqttConnectOptions.MQTT_VERSION_3_1),
+new
AllowableValue(String.valueOf(MqttVersion.MQTT_VERSION_3_1.getNumericValue()),
"v3.1.0");
+
+public enum MqttVersion {
+MQTT_VERSION_3_1(3),
+MQTT_VERSION_3_1_1(4),
+MQTT_VERSION_5_0(5);
+
+private final int numericValue;
+
+MqttVersion(int numericValue) {
+this.numericValue = numericValue;
+}
+
+public int getNumericValue() {
+return numericValue;
+}
+}
+
+public enum SupportedSchemes {
+TCP,
+SSL,
+WS,
+WSS;
+
+public static String getValuesAsString(String delimiter) {
+return String.join(delimiter, Arrays.stream(values()).map(value ->
value.name().toLowerCase()).toArray(String[]::new));
+}
+}
Review Comment:
Changed and renamed based on the review of @turcsanyip .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952880574
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/NifiMqttClient.java:
##
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.common;
+
+public interface NifiMqttClient {
Review Comment:
I went with MqttClient without prefix. Hope it is good that way!
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/common/NifiMqttMessage.java:
##
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.common;
+
+public class NifiMqttMessage {
Review Comment:
Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952879664
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.adapters;
+
+import com.hivemq.client.mqtt.datatypes.MqttQos;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5Client;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5Connect;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5ConnectBuilder;
+import org.apache.nifi.processors.mqtt.common.MqttConnectionProperties;
+import org.apache.nifi.processors.mqtt.common.NifiMqttCallback;
+import org.apache.nifi.processors.mqtt.common.NifiMqttClient;
+import org.apache.nifi.processors.mqtt.common.NifiMqttException;
+import org.apache.nifi.processors.mqtt.common.NifiMqttMessage;
+
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.TrustManagerFactory;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.StandardCharsets;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.cert.CertificateException;
+import java.util.Arrays;
+import java.util.Objects;
+
+public class HiveMqV5ClientAdapter implements NifiMqttClient {
+
+private final Mqtt5Client mqtt5Client;
+
+private NifiMqttCallback callback;
+
+public HiveMqV5ClientAdapter(Mqtt5BlockingClient mqtt5BlockingClient) {
+this.mqtt5Client = mqtt5BlockingClient;
+}
+
+@Override
+public boolean isConnected() {
+return mqtt5Client.getState().isConnected();
+}
+
+@Override
+public void connect(MqttConnectionProperties connectionProperties) throws
NifiMqttException {
+final Mqtt5ConnectBuilder connectBuilder = Mqtt5Connect.builder()
+.keepAlive(connectionProperties.getKeepAliveInterval());
+
+final boolean cleanSession = connectionProperties.isCleanSession();
+connectBuilder.cleanStart(cleanSession);
+if (!cleanSession) {
+
connectBuilder.sessionExpiryInterval(connectionProperties.getSessionExpiryInterval());
+}
+
+final String lastWillTopic = connectionProperties.getLastWillTopic();
+if (lastWillTopic != null) {
+connectBuilder.willPublish()
+.topic(lastWillTopic)
+
.payload(connectionProperties.getLastWillMessage().getBytes())
+.retain(connectionProperties.getLastWillRetain())
+
.qos(MqttQos.fromCode(connectionProperties.getLastWillQOS()))
+.applyWillPublish();
+}
+
+// checking for presence of password because username can be null
+final char[] password = connectionProperties.getPassword();
+if (password != null) {
+connectBuilder.simpleAuth()
+.username(connectionProperties.getUsername())
+.password(toBytes(password))
+.applySimpleAuth();
+
+clearSensitive(connectionProperties.getPassword());
+clearSensitive(password);
+}
+
+final Mqtt5Connect mqtt5Connect = connectBuilder.build();
+mqtt5Client.toBlocking().connect(mqtt5Connect);
+}
+
+@Override
+public void disconnect(long disconnectTimeout) throws NifiMqttException {
+// Currently it is not possible to set timeout for disconnect with
HiveMQ Client. (Only connect timeout exists.)
+mqtt5Client.toBlocking().disconnect();
+}
+
+@Override
+public void close() throws NifiMqttException {
+// there is no paho's close equivalent in hivemq client
+}
+
+@Override
+public void publish(String topic, NifiMqttMessage message) throws
NifiMqttException {
+mqtt5Client.toAsync().publishWith()
+.topic(topic)
+
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6225: NIFI-10251 Add v5 protocol support for existing MQTT processors
nandorsoma commented on code in PR #6225:
URL: https://github.com/apache/nifi/pull/6225#discussion_r952879128
##
nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/adapters/HiveMqV5ClientAdapter.java:
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.mqtt.adapters;
+
+import com.hivemq.client.mqtt.datatypes.MqttQos;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient;
+import com.hivemq.client.mqtt.mqtt5.Mqtt5Client;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5Connect;
+import com.hivemq.client.mqtt.mqtt5.message.connect.Mqtt5ConnectBuilder;
+import org.apache.nifi.processors.mqtt.common.MqttConnectionProperties;
+import org.apache.nifi.processors.mqtt.common.NifiMqttCallback;
+import org.apache.nifi.processors.mqtt.common.NifiMqttClient;
+import org.apache.nifi.processors.mqtt.common.NifiMqttException;
+import org.apache.nifi.processors.mqtt.common.NifiMqttMessage;
+
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.TrustManagerFactory;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
+import java.nio.charset.StandardCharsets;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.cert.CertificateException;
+import java.util.Arrays;
+import java.util.Objects;
+
+public class HiveMqV5ClientAdapter implements NifiMqttClient {
+
+private final Mqtt5Client mqtt5Client;
+
+private NifiMqttCallback callback;
+
+public HiveMqV5ClientAdapter(Mqtt5BlockingClient mqtt5BlockingClient) {
+this.mqtt5Client = mqtt5BlockingClient;
+}
+
+@Override
+public boolean isConnected() {
+return mqtt5Client.getState().isConnected();
+}
+
+@Override
+public void connect(MqttConnectionProperties connectionProperties) throws
NifiMqttException {
+final Mqtt5ConnectBuilder connectBuilder = Mqtt5Connect.builder()
+.keepAlive(connectionProperties.getKeepAliveInterval());
+
+final boolean cleanSession = connectionProperties.isCleanSession();
+connectBuilder.cleanStart(cleanSession);
+if (!cleanSession) {
+
connectBuilder.sessionExpiryInterval(connectionProperties.getSessionExpiryInterval());
+}
+
+final String lastWillTopic = connectionProperties.getLastWillTopic();
+if (lastWillTopic != null) {
+connectBuilder.willPublish()
+.topic(lastWillTopic)
+
.payload(connectionProperties.getLastWillMessage().getBytes())
+.retain(connectionProperties.getLastWillRetain())
+
.qos(MqttQos.fromCode(connectionProperties.getLastWillQOS()))
+.applyWillPublish();
+}
+
+// checking for presence of password because username can be null
+final char[] password = connectionProperties.getPassword();
+if (password != null) {
+connectBuilder.simpleAuth()
+.username(connectionProperties.getUsername())
+.password(toBytes(password))
+.applySimpleAuth();
+
+clearSensitive(connectionProperties.getPassword());
+clearSensitive(password);
+}
+
+final Mqtt5Connect mqtt5Connect = connectBuilder.build();
+mqtt5Client.toBlocking().connect(mqtt5Connect);
+}
+
+@Override
+public void disconnect(long disconnectTimeout) throws NifiMqttException {
+// Currently it is not possible to set timeout for disconnect with
HiveMQ Client. (Only connect timeout exists.)
+mqtt5Client.toBlocking().disconnect();
+}
+
+@Override
+public void close() throws NifiMqttException {
+// there is no paho's close equivalent in hivemq client
+}
+
+@Override
+public void publish(String topic, NifiMqttMessage message) throws
NifiMqttException {
+mqtt5Client.toAsync().publishWith()
+.topic(topic)
+
[jira] [Resolved] (NIFI-5232) HttpConnectionService controller service
[ https://issues.apache.org/jira/browse/NIFI-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen resolved NIFI-5232. Resolution: Duplicate [~exceptionfactory] yeah, I agree that your ticket resolved this. Thanks for bringing that up. I think adding your component to the scripting package might be a good idea for 1.18 because I've seen a lot of cases on our team where it might be helpful to script out some HTTP calls and really go to town on the responses (government systems, nasty output a lot of the time). > HttpConnectionService controller service > > > Key: NIFI-5232 > URL: https://issues.apache.org/jira/browse/NIFI-5232 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Priority: Major > > The functionality of InvokeHttp and related processors should be copied over > to a controller service that can do much the same thing. This controller > service would be able to handle all of the common scenarios with HTTP > connections from processors going forward. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10348) Upgrade Tomcat Embed to 8.5.82 for Flume Processors
[ https://issues.apache.org/jira/browse/NIFI-10348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Gough updated NIFI-10348: Fix Version/s: 1.18.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Tomcat Embed to 8.5.82 for Flume Processors > --- > > Key: NIFI-10348 > URL: https://issues.apache.org/jira/browse/NIFI-10348 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.18.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Apache Flume Processors have a transitive dependency on Apache Tomcat Embed > Core 8.5.46, which has several associated vulnerabilities. Although most of > these vulnerabilities relate to Apache Tomcat Server, upgrading a more recent > version of the library includes several bug fixes and resolves false > positives. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10348) Upgrade Tomcat Embed to 8.5.82 for Flume Processors
[ https://issues.apache.org/jira/browse/NIFI-10348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583745#comment-17583745 ] ASF subversion and git services commented on NIFI-10348: Commit 90aa778a6ca4723623465709222e66d5cc5cdbc2 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=90aa778a6c ] NIFI-10348 Upgraded Tomcat Embed to 8.5.82 for Flume NAR - Upgraded tomcat-embed-core from 8.5.46 to 8.5.82 for Flume components Signed-off-by: Nathan Gough This closes #6292. > Upgrade Tomcat Embed to 8.5.82 for Flume Processors > --- > > Key: NIFI-10348 > URL: https://issues.apache.org/jira/browse/NIFI-10348 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > Apache Flume Processors have a transitive dependency on Apache Tomcat Embed > Core 8.5.46, which has several associated vulnerabilities. Although most of > these vulnerabilities relate to Apache Tomcat Server, upgrading a more recent > version of the library includes several bug fixes and resolves false > positives. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog closed pull request #6292: NIFI-10348 Upgrade Tomcat Embed to 8.5.82 for Flume NAR
thenatog closed pull request #6292: NIFI-10348 Upgrade Tomcat Embed to 8.5.82 for Flume NAR URL: https://github.com/apache/nifi/pull/6292 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #6292: NIFI-10348 Upgrade Tomcat Embed to 8.5.82 for Flume NAR
thenatog commented on PR #6292: URL: https://github.com/apache/nifi/pull/6292#issuecomment-1224321939 Verified that all dependencies for tomcat-embed-core are at least 8.5.82. +1 will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6291: NIFI-10347 Upgrade Metrics-graphite to 4.2.10
exceptionfactory commented on PR #6291: URL: https://github.com/apache/nifi/pull/6291#issuecomment-1224296068 Thanks for the vulnerability reference @mr1716. CVE-2016-4000 applies to Jython, which is a test dependency of `metrics-graphite`, but is not included at runtime in any distributions of Apache NiFi. Upgrading to the latest version should be provide other improvements. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mr1716 commented on pull request #6291: NIFI-10347 Upgrade Metrics-graphite to 4.2.10
mr1716 commented on PR #6291: URL: https://github.com/apache/nifi/pull/6291#issuecomment-1224288461 This does resolve CVE [CVE-2016-4000](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-4000) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9878) DistributedCacheMap Handshake failure, processor hang indefinitely.
[ https://issues.apache.org/jira/browse/NIFI-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583714#comment-17583714 ] Jon Shoemaker commented on NIFI-9878: - Experiencing the same issue. In our scenario it works correctly most of the time but occasionally the handshake response is never received and the processor thread hangs until the processor is terminated. Some of these stuck threads happen when the DistributedCacheServer is restarted. > DistributedCacheMap Handshake failure, processor hang indefinitely. > --- > > Key: NIFI-9878 > URL: https://issues.apache.org/jira/browse/NIFI-9878 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.15.3 >Reporter: Aaron Rich >Priority: Major > Labels: Handshake, distributed_cache > Attachments: image-2022-04-05-21-54-31-002.png, > image-2022-04-05-21-55-16-221.png > > > When a DistributedCacheMapClient attempts to connect to a > DistributedCacheMapServer, but the handshake response is never received by > the client, the PutDistributedCacheMap processor with hang indefinitely. The > handshake never times out. > A situation like this can be caused if a proxy allows for the TCP connection > to be established between client and server but fails to deliver handshake > data to/from DistributedCacheMapServer (for example an unstable Istio service > mesh between the two). Could also happen if a client was accidentally > misconfigured to point to wrong TCP server point (one that wasn't hosting a > DistributedCacheMapServer. > Steps to recreate: > 1) Set up a PutDistributedCacheMap processor with a > DistributedMapCacheClientService > 2) Configure DistributedMapCacheClientService to point to a non > DistributedCacheMapServer tcp server (nc -lk 127.0.0.1 4457). This simulates > a situation where the socket connection can be made but there is no handshake > response from the server (for example, server is in bad state and unable to > respond, a proxy is misbehaving, etc). > 3) use generateFlowFile to trigger PutDistributedCacheMap processor. > 4) processor will hang with no failure or success. Processor will have to be > force terminated. > !image-2022-04-05-21-54-31-002.png! > !image-2022-04-05-21-55-16-221.png! > Hang occurs at : > CacheClientRequestHandler.java:92: handshakeHandler.waitHandshakeComplete(); > > Currently, the "connection timeout" parameter is only used to timeout the > establishment of the TCP socket connection, not the full application layer > connection. > Suggestion: > Handshake should have a timeout too to be robust to handle a network outage > where the TCP connection is able to be created, but the handshake data can't > be exchanged. The processor hanging prevents any way to handle this error in > a dataflow. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog commented on pull request #6292: NIFI-10348 Upgrade Tomcat Embed to 8.5.82 for Flume NAR
thenatog commented on PR #6292: URL: https://github.com/apache/nifi/pull/6292#issuecomment-1224269879 Will review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10346) Update OWASP Dependency Check Suppressions
[ https://issues.apache.org/jira/browse/NIFI-10346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10346: Fix Version/s: 1.18.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Update OWASP Dependency Check Suppressions > -- > > Key: NIFI-10346 > URL: https://issues.apache.org/jira/browse/NIFI-10346 > Project: Apache NiFi > Issue Type: Task > Components: Documentation Website >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.18.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The OWASP Dependency Check Plugin version 7.1.1 marks several libraries as > vulnerable when the vulnerability applies to server components, but not > client components. In other cases, the plugin associates vulnerabilities with > a different product based on similar naming. The Suppressions configuration > should be updated to note and suppress these findings. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6310: NIFI-10366: Make Default Run Duration configurable
exceptionfactory commented on code in PR #6310:
URL: https://github.com/apache/nifi/pull/6310#discussion_r952801009
##
nifi-api/src/main/java/org/apache/nifi/annotation/behavior/DefaultRunDuration.java:
##
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.annotation.behavior;
+
+import java.time.Duration;
+
+public enum DefaultRunDuration {
+NO_BATCHING(Duration.ZERO),
+TWENTY_FIVE_MILLIS(Duration.ofMillis(25)),
+FIFTY_MILLIS(Duration.ofMillis(50)),
+ONE_HUNDRED_MILLIS(Duration.ofMillis(100)),
+TWO_HUNDRED_FIFTY_MILLIS(Duration.ofMillis(250)),
+FIVE_HUNDRED_MILLIS(Duration.ofMillis(500)),
+ONE_SECOND(Duration.ofSeconds(1)),
+TWO_SECONDS(Duration.ofSeconds(2));
+
+private final Duration duration;
+
+DefaultRunDuration(final Duration duration) {
+this.duration = duration;
+}
+
+public Duration getDuration() {
+return Duration.ofMillis(1000);
Review Comment:
This needs to be changed to return the `duration` property.
```suggestion
return duration;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mr1716 opened a new pull request, #6326: NIFI-10388 Upgrade xmlunit-matchers to 2.9.0
mr1716 opened a new pull request, #6326: URL: https://github.com/apache/nifi/pull/6326 # Summary [NIFI-10388](https://issues.apache.org/jira/browse/NIFI-10388) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [NIFI-10388](https://issues.apache.org/jira/browse/NIFI-10388) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10388) Upgrade xmlunit-matchers to 2.9.0 to eliminate CVEs
Mike R created NIFI-10388: - Summary: Upgrade xmlunit-matchers to 2.9.0 to eliminate CVEs Key: NIFI-10388 URL: https://issues.apache.org/jira/browse/NIFI-10388 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.17.0 Reporter: Mike R xmlunit-matchers to 2.9.0, from the current 2.6.3 to eliminate vulnerability from CVE [CVE-2020-15250|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15250] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1367: MINIFICPP-1822 - Add alert capability
szaszm commented on code in PR #1367:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1367#discussion_r952772655
##
libminifi/src/core/logging/alert/AlertSink.cpp:
##
@@ -0,0 +1,268 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "core/logging/alert/AlertSink.h"
+#include "core/TypedValues.h"
+#include "core/ClassLoader.h"
+#include "utils/HTTPClient.h"
+#include "utils/Hash.h"
+#include "core/logging/Utils.h"
+#include "controllers/SSLContextService.h"
+
+#include "rapidjson/rapidjson.h"
+#include "rapidjson/document.h"
+#include "rapidjson/stringbuffer.h"
+#include "rapidjson/writer.h"
+
+namespace org::apache::nifi::minifi::core::logging {
+
+AlertSink::AlertSink(Config config, std::shared_ptr logger)
+: config_(std::move(config)),
+ live_logs_(config_.rate_limit),
+ buffer_(config_.buffer_limit, config_.batch_size),
+ logger_(std::move(logger)) {
+ set_level(config_.level);
+ next_flush_ = clock_->timeSinceEpoch() + config_.flush_period;
+ flush_thread_ = std::thread([this] {run();});
+}
+
+std::shared_ptr AlertSink::create(const std::string&
prop_name_prefix, const std::shared_ptr& logger_properties,
std::shared_ptr logger) {
+ Config config;
+
+ if (auto url = logger_properties->getString(prop_name_prefix + ".url")) {
+config.url = url.value();
+ } else {
+logger->log_info("Missing '%s.url' value, network logging won't be
available", prop_name_prefix);
+return {};
+ }
+
+ if (auto filter_str = logger_properties->getString(prop_name_prefix +
".filter")) {
+try {
+ config.filter = utils::Regex{filter_str.value()};
+} catch (const std::regex_error& err) {
+ logger->log_error("Invalid '%s.filter' value, network logging won't be
available: %s", prop_name_prefix, err.what());
+ return {};
+}
+ } else {
+logger->log_error("Missing '%s.filter' value, network logging won't be
available", prop_name_prefix);
+return {};
+ }
+
+ auto readPropertyOr = [&] (auto suffix, auto parser, auto fallback) {
+if (auto prop_str = logger_properties->getString(prop_name_prefix +
suffix)) {
+ if (auto prop_val = parser(prop_str.value())) {
+return prop_val.value();
+ }
+ logger->log_error("Invalid '%s' value, using default '%s'",
prop_name_prefix + suffix, fallback);
+} else {
+ logger->log_info("Missing '%s' value, using default '%s'",
prop_name_prefix + suffix, fallback);
+}
+return parser(fallback).value();
+ };
+
+ auto datasize_parser = [] (const std::string& str) -> std::optional {
+int val;
+if (DataSizeValue::StringToInt(str, val)) {
+ return val;
+}
+return {};
+ };
+
+ config.batch_size = readPropertyOr(".batch.size", datasize_parser, "100 KB");
+ config.flush_period = readPropertyOr(".flush.period",
TimePeriodValue::fromString, "5 s").getMilliseconds();
+ config.rate_limit = readPropertyOr(".rate.limit",
TimePeriodValue::fromString, "10 min").getMilliseconds();
+ config.buffer_limit = readPropertyOr(".buffer.limit", datasize_parser, "1
MB");
+ config.level = readPropertyOr(".level", utils::parse_log_level, "trace");
+ config.ssl_service_name = logger_properties->getString(prop_name_prefix +
".ssl.context.service");
+
+ return std::shared_ptr(new AlertSink(std::move(config),
std::move(logger)));
Review Comment:
nevermind, shared_ptr constructor frees the ptr before throwing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-9374) Implement Standard Logging for Deprecated Features
[ https://issues.apache.org/jira/browse/NIFI-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583677#comment-17583677 ] ASF subversion and git services commented on NIFI-9374: --- Commit fa85a05a2ba9ed7c9c613f880efaf53ca720c2f8 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fa85a05a2b ] NIFI-9374 Added Deprecation Logger - Added nifi-deprecation-log module with interface and implementation using SLF4J - Updated standard logback.xml with nifi-deprecation.log appender - Updated NiFiLegacyCipherProvider with deprecation logging - Set Size, Time Policy, and Total Size Limit for Deprecation Log This closes #6300 Signed-off-by: Paul Grey > Implement Standard Logging for Deprecated Features > -- > > Key: NIFI-9374 > URL: https://issues.apache.org/jira/browse/NIFI-9374 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > Over the course of multiple releases and numerous capability improvements, > various areas of the code have become deprecated. Deprecated classes, > methods, and properties are required to maintain compatibility when upgrading > between minor releases, but it can be difficult for both developers and users > to track all uses of deprecated features. > Following the pattern of other libraries and frameworks, the purpose of this > issue is to implement standard logging for deprecated features. The default > logging configuration should include a new log file dedicated to deprecated > usage patterns. The implementation should wrap standard SLF4J logging and > provide a common standard for logger names. This approach should make it easy > to add deprecation log warnings throughout the application, and also make it > easy for users to determine whether their configuration is using deprecated > features. > Having a dedicated log for deprecated features will provide a straightforward > method of determining whether a given NiFi installation can be upgraded to > future major releases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 closed pull request #6300: NIFI-9374 Add Standard Deprecation Logger
greyp9 closed pull request #6300: NIFI-9374 Add Standard Deprecation Logger URL: https://github.com/apache/nifi/pull/6300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10382) Upgrade Flume to 1.10.1
[ https://issues.apache.org/jira/browse/NIFI-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-10382: -- Fix Version/s: 1.18.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Flume to 1.10.1 > --- > > Key: NIFI-10382 > URL: https://issues.apache.org/jira/browse/NIFI-10382 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.18.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Apache Flume JMS Sources in versions prior to 1.10.1 are vulnerable to remote > code execution under limited conditions where an attacker controls a remote > LDAP server as described in > [CVE-2022-34916|https://cve.mitre.org/cgi-bin/cvename.cgi?name=2022-34916]. > Dependencies on Apache Flume libraries should be upgraded to 1.10.1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10382) Upgrade Flume to 1.10.1
[ https://issues.apache.org/jira/browse/NIFI-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583671#comment-17583671 ] ASF subversion and git services commented on NIFI-10382: Commit ca991a680560a72999858a102a7b1f9e65e721e4 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ca991a6805 ] NIFI-10382 Upgraded Flume dependencies from 1.10.0 to 1.10.1 Signed-off-by: Pierre Villard This closes #6320. > Upgrade Flume to 1.10.1 > --- > > Key: NIFI-10382 > URL: https://issues.apache.org/jira/browse/NIFI-10382 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 10m > Remaining Estimate: 0h > > Apache Flume JMS Sources in versions prior to 1.10.1 are vulnerable to remote > code execution under limited conditions where an attacker controls a remote > LDAP server as described in > [CVE-2022-34916|https://cve.mitre.org/cgi-bin/cvename.cgi?name=2022-34916]. > Dependencies on Apache Flume libraries should be upgraded to 1.10.1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #6320: NIFI-10382 Upgrade Flume dependencies from 1.10.0 to 1.10.1
asfgit closed pull request #6320: NIFI-10382 Upgrade Flume dependencies from 1.10.0 to 1.10.1 URL: https://github.com/apache/nifi/pull/6320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request, #1403: MINIFICPP-1906 Restore original timeout for CivetServer
lordgamez opened a new pull request, #1403: URL: https://github.com/apache/nifi-minifi-cpp/pull/1403 There was a change in `CivetWeb` (commit hash: `a7361235`) where a previously used `poll` function with constant timeout of 200 ms was replaced with the `mg_poll` function. This function uses the SOCKET_TIMEOUT_QUANTUM constant for the timeout value of a single poll call. The default value of this constant is 2000 ms and due to this the shutdown of the `CivetServer` had to wait 10 times more for the ongoing poll to finish in every single test case, which increased the runtime of the test cases. The SOCKET_TIMEOUT_QUANTUM is changed back to 200 ms in this PR. https://issues.apache.org/jira/browse/MINIFICPP-1906 -- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952747264
##
libminifi/include/utils/ResourceQueue.h:
##
@@ -0,0 +1,125 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "core/logging/Logger.h"
+#include "concurrentqueue.h"
+#include "MinifiConcurrentQueue.h"
+
+namespace org::apache::nifi::minifi::utils {
+
+template
+class ResourceQueue : public
std::enable_shared_from_this> {
Review Comment:
good idea, I've added some explanatory comments in
https://github.com/apache/nifi-minifi-cpp/commit/3edf5fcfd603463b67bd6c072fa0128bbb861787#diff-4205c4389dbd9a8dee16d4d38da02ada57c19d038893abdcb3f29424c7d1dd86R36-R42
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6300: NIFI-9374 Add Standard Deprecation Logger
exceptionfactory commented on PR #6300: URL: https://github.com/apache/nifi/pull/6300#issuecomment-1224203090 Thanks for the thorough testing @greyp9, great recommendation on the `totalSizeCap` property! I pushed an update add the property with a setting of `100MB`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952746080
##
extensions/http-curl/client/HTTPClient.h:
##
@@ -270,48 +249,33 @@ class HTTPClient : public BaseHTTPClient, public
core::Connectable {
void configure_secure_connection(CURL *http_session);
- bool isSecure(const std::string );
+ std::chrono::milliseconds getAbsoluteTimeout() const { return
3*read_timeout_; }
- HTTPReadCallback content_;
+ utils::HTTPReadCallback content_{std::numeric_limits::max()};
std::shared_ptr ssl_context_service_;
std::string url_;
- std::chrono::milliseconds connect_timeout_ms_{3};
- // read timeout.
- std::chrono::milliseconds read_timeout_ms_{3};
- char *content_type_str_{nullptr};
- std::string content_type_;
- struct curl_slist *headers_{nullptr};
- HTTPReadCallback *callback{nullptr};
- HTTPUploadCallback *write_callback_{nullptr};
- int64_t http_code_{0};
- ByteOutputCallback read_callback_{INT_MAX};
- utils::HTTPHeaderResponse header_response_{-1};
-
- CURLcode res{CURLE_OK};
-
- CURL* http_session_{nullptr};
- curl_mime* form_{nullptr};
-
std::string method_;
- std::chrono::milliseconds keep_alive_probe_{-1};
+ std::chrono::milliseconds connect_timeout_{std::chrono::seconds(30)};
+ std::chrono::milliseconds read_timeout_{std::chrono::seconds(30)};
- std::chrono::milliseconds keep_alive_idle_{-1};
+ HTTPResponseData response_data_;
- struct BasicAuthCredentials {
-BasicAuthCredentials(std::string username, std::string password) :
username(std::move(username)), password(std::move(password)) {}
+ CURLcode res_{CURLE_OK};
-std::string username;
-std::string password;
- };
+ RequestHeaders request_headers_;
- std::optional username_password_;
+ std::unique_ptr http_session_{nullptr,
curl_easy_cleanup};
+ std::unique_ptr form_{nullptr,
curl_mime_free};
Review Comment:
good idea, I've replaced them in
https://github.com/apache/nifi-minifi-cpp/commit/3edf5fcfd603463b67bd6c072fa0128bbb861787#diff-475912c726caf957fa46ce1b55d3e845e89aa19b4c7caf2cd4a0ae1f860cbdd5R241-R242
Didnt remove the include from the header because there is a member that
still depends on it `CURLcode res_{CURLE_OK}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952744327
##
extensions/http-curl/processors/InvokeHTTP.cpp:
##
@@ -46,31 +45,31 @@ namespace org::apache::nifi::minifi::processors {
std::string InvokeHTTP::DefaultContentType = "application/octet-stream";
const core::Property InvokeHTTP::Method("HTTP Method", "HTTP request method
(GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS). "
- "Arbitrary methods are also supported.
Methods other than POST, PUT and PATCH will be sent without a message body.",
- "GET");
+ "Arbitrary methods are
also supported. Methods other than POST, PUT and PATCH will be sent without a
message body.",
+"GET");
Review Comment:
fixed the indentation issues in
https://github.com/apache/nifi-minifi-cpp/commit/3edf5fcfd603463b67bd6c072fa0128bbb861787
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952743952
##
extensions/http-curl/client/RequestHeaders.h:
##
@@ -0,0 +1,58 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+
+#ifdef WIN32
+#pragma comment(lib, "wldap32.lib" )
+#pragma comment(lib, "crypt32.lib" )
+#pragma comment(lib, "Ws2_32.lib")
+
+#define CURL_STATICLIB
+#include
+#else
+#include
+#endif
+
+namespace org::apache::nifi::minifi::extensions::curl {
+class RequestHeaders {
+ public:
+ RequestHeaders() = default;
+
+ void appendHeader(std::string key, std::string value);
+
+ void disableExpectHeader();
+
+ [[nodiscard]] std::unique_ptr get() const;
+ [[nodiscard]] auto size() const { return headers_.size(); }
+ [[nodiscard]] bool empty() const;
+
+ std::string& operator[](const std::string& key);
+ std::string& operator[](std::string&& key);
+
+ [[nodiscard]] bool contains(const std::string& key) const;
+ void erase(const std::string& key);
+
+ private:
+ std::unordered_map headers_;
+};
Review Comment:
replaced it in
https://github.com/apache/nifi-minifi-cpp/pull/1383/commits/176864bc890da609911197adcc314b61d91e624b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10387) Scripted components incorrectly marking whether a script needs to be reloaded
[ https://issues.apache.org/jira/browse/NIFI-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-10387: Status: Patch Available (was: In Progress) > Scripted components incorrectly marking whether a script needs to be reloaded > - > > Key: NIFI-10387 > URL: https://issues.apache.org/jira/browse/NIFI-10387 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > In many of the scripted components (InvokeScriptedProcessor, scripted > controller services, etc.) there is a negative logic bug when setting the > scriptNeedsReload variable to indicate that the script should be reloaded. > Specifically, if the script does not load successfully, scriptNeedsReload is > set false when it should be true. In most/all cases, this error does not > manifest because there is also a check to make sure there is an actual > lookupService object retrieved from the (successfully loaded) script. However > there may be situations in which this negative logic bug could manifest and > thus it should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on pull request #6300: NIFI-9374 Add Standard Deprecation Logger
greyp9 commented on PR #6300:
URL: https://github.com/apache/nifi/pull/6300#issuecomment-1224193543
> Thanks for the feedback regarding the Logback configuration @greyp9!
>
> I updated the Logback configuration to set `additivity="false"` for the
`deprecation` logger to avoid duplicating messages in `nifi-app.log`. I also
updated the `DEPRECATION_FILE` appender to set a maximum file size of 10 MB and
a maximum history of 10. Although the log could produce many duplicate
messages, this configuration limits the amount of space that could be consumed
in the default configuration.
Tried this out again. The additional config does not seem to be sufficient
to limit the deprecation logger output. This configuration seems to achieve
the intent:
```
${org.apache.nifi.bootstrap.config.log.dir}/nifi-deprecation_%d.%i.log
10MB
10
100MB
```
This may be a corner case, due to logback being fed logging events so
quickly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mattyb149 opened a new pull request, #6325: NIFI-10387: Fixed negative logic bug in scripted components
mattyb149 opened a new pull request, #6325: URL: https://github.com/apache/nifi/pull/6325 # Summary [NIFI-10387](https://issues.apache.org/jira/browse/NIFI-10387) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-10387) Scripted components incorrectly marking whether a script needs to be reloaded
[ https://issues.apache.org/jira/browse/NIFI-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-10387: --- Assignee: Matt Burgess > Scripted components incorrectly marking whether a script needs to be reloaded > - > > Key: NIFI-10387 > URL: https://issues.apache.org/jira/browse/NIFI-10387 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > In many of the scripted components (InvokeScriptedProcessor, scripted > controller services, etc.) there is a negative logic bug when setting the > scriptNeedsReload variable to indicate that the script should be reloaded. > Specifically, if the script does not load successfully, scriptNeedsReload is > set false when it should be true. In most/all cases, this error does not > manifest because there is also a check to make sure there is an actual > lookupService object retrieved from the (successfully loaded) script. However > there may be situations in which this negative logic bug could manifest and > thus it should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10387) Scripted components incorrectly marking whether a script needs to be reloaded
Matt Burgess created NIFI-10387: --- Summary: Scripted components incorrectly marking whether a script needs to be reloaded Key: NIFI-10387 URL: https://issues.apache.org/jira/browse/NIFI-10387 Project: Apache NiFi Issue Type: Bug Reporter: Matt Burgess In many of the scripted components (InvokeScriptedProcessor, scripted controller services, etc.) there is a negative logic bug when setting the scriptNeedsReload variable to indicate that the script should be reloaded. Specifically, if the script does not load successfully, scriptNeedsReload is set false when it should be true. In most/all cases, this error does not manifest because there is also a check to make sure there is an actual lookupService object retrieved from the (successfully loaded) script. However there may be situations in which this negative logic bug could manifest and thus it should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1402: MINIFICPP-1919 centos HTTPClientTests fix
martinzink commented on code in PR #1402: URL: https://github.com/apache/nifi-minifi-cpp/pull/1402#discussion_r952712733 ## docker/centos/Dockerfile: ## @@ -38,7 +38,7 @@ ENV USER minificpp # gpsd-devel is in EPEL RUN yum -y install epel-release && yum -y install java-1.8.0-openjdk java-1.8.0-openjdk-devel python36-devel sudo git which maven make libarchive boost-devel lua-devel libusbx-devel libpng-devel \ -gpsd-devel libpcap-devel ccache +gpsd-devel libpcap-devel ccache ca-certificates Review Comment: ca-certs were expired on the centos 7 docker image -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink opened a new pull request, #1402: MINIFICPP-1919 centos HTTPClientTests fix
martinzink opened a new pull request, #1402: URL: https://github.com/apache/nifi-minifi-cpp/pull/1402 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically main)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file? - [X] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-1919) HTTPClientTests fails on centos
Martin Zink created MINIFICPP-1919: -- Summary: HTTPClientTests fails on centos Key: MINIFICPP-1919 URL: https://issues.apache.org/jira/browse/MINIFICPP-1919 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Martin Zink Assignee: Martin Zink SSL without SSLContextService test fails on centos due to expired ca certs on centos 7. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952697746
##
extensions/http-curl/client/HTTPClient.h:
##
@@ -51,17 +53,28 @@
#include "core/logging/Logger.h"
#include "core/logging/LoggerConfiguration.h"
-namespace org::apache::nifi::minifi::utils {
+namespace org::apache::nifi::minifi::extensions::curl {
-/**
- * Purpose and Justification: Pull the basics for an HTTPClient into a self
contained class. Simply provide
- * the URL and an SSLContextService ( can be null).
- *
- * Since several portions of the code have been relying on curl, we can
encapsulate most CURL HTTP
- * operations here without maintaining it everywhere. Further, this will help
with testing as we
- * only need to to test our usage of CURL once
- */
-class HTTPClient : public BaseHTTPClient, public core::Connectable {
+struct KeepAliveProbeData {
+ std::chrono::seconds keep_alive_delay;
+ std::chrono::seconds keep_alive_interval;
+};
+
+struct HTTPResponseData {
+ std::vector response_body;
+ utils::HTTPHeaderResponse header_response;
+ char* response_content_type;
+ int64_t response_code;
Review Comment:
not at all, fixed in
https://github.com/apache/nifi-minifi-cpp/pull/1383/commits/3edf5fcfd603463b67bd6c072fa0128bbb861787#diff-475912c726caf957fa46ce1b55d3e845e89aa19b4c7caf2cd4a0ae1f860cbdd5R56-R57
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952696800
##
extensions/http-curl/client/HTTPClient.cpp:
##
@@ -164,212 +161,191 @@ bool HTTPClient::setSpecificSSLVersion(SSLVersion
specific_version) {
}
// If not set, the default will be TLS 1.0, see
https://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html
-bool HTTPClient::setMinimumSSLVersion(SSLVersion minimum_version) {
+bool HTTPClient::setMinimumSSLVersion(utils::SSLVersion minimum_version) {
CURLcode ret = CURLE_UNKNOWN_OPTION;
switch (minimum_version) {
-case SSLVersion::TLSv1_0:
- ret = curl_easy_setopt(http_session_, CURLOPT_SSLVERSION,
CURL_SSLVERSION_TLSv1_0);
+case utils::SSLVersion::TLSv1_0:
+ ret = curl_easy_setopt(http_session_.get(), CURLOPT_SSLVERSION,
CURL_SSLVERSION_TLSv1_0);
break;
-case SSLVersion::TLSv1_1:
- ret = curl_easy_setopt(http_session_, CURLOPT_SSLVERSION,
CURL_SSLVERSION_TLSv1_1);
+case utils::SSLVersion::TLSv1_1:
+ ret = curl_easy_setopt(http_session_.get(), CURLOPT_SSLVERSION,
CURL_SSLVERSION_TLSv1_1);
break;
-case SSLVersion::TLSv1_2:
- ret = curl_easy_setopt(http_session_, CURLOPT_SSLVERSION,
CURL_SSLVERSION_TLSv1_2);
+case utils::SSLVersion::TLSv1_2:
+ ret = curl_easy_setopt(http_session_.get(), CURLOPT_SSLVERSION,
CURL_SSLVERSION_TLSv1_2);
break;
}
return ret == CURLE_OK;
}
-DEPRECATED(/*deprecated in*/ 0.8.0, /*will remove in */ 2.0) void
HTTPClient::setConnectionTimeout(int64_t timeout) {
- setConnectionTimeout(std::chrono::milliseconds(timeout * 1000));
-}
-
-DEPRECATED(/*deprecated in*/ 0.8.0, /*will remove in */ 2.0) void
HTTPClient::setReadTimeout(int64_t timeout) {
- setReadTimeout(std::chrono::milliseconds(timeout * 1000));
-}
-
void HTTPClient::setConnectionTimeout(std::chrono::milliseconds timeout) {
- connect_timeout_ms_ = timeout;
+ if (timeout < 0ms) {
+logger_->log_error("Invalid timeout");
+return;
+ }
+ connect_timeout_ = timeout;
}
void HTTPClient::setReadTimeout(std::chrono::milliseconds timeout) {
- read_timeout_ms_ = timeout;
+ if (timeout < 0ms) {
+logger_->log_error("Invalid timeout");
+return;
+ }
+ read_timeout_ = timeout;
}
-void HTTPClient::setReadCallback(HTTPReadCallback *callbackObj) {
- callback = callbackObj;
- curl_easy_setopt(http_session_, CURLOPT_WRITEFUNCTION,
::HTTPRequestResponse::recieve_write);
- curl_easy_setopt(http_session_, CURLOPT_WRITEDATA,
static_cast(callbackObj));
+void HTTPClient::setReadCallback(std::unique_ptr&&
callback) {
Review Comment:
same here: I fixed it in
https://github.com/apache/nifi-minifi-cpp/pull/1383/commits/3edf5fcfd603463b67bd6c072fa0128bbb861787#diff-475912c726caf957fa46ce1b55d3e845e89aa19b4c7caf2cd4a0ae1f860cbdd5R100
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952697238
##
extensions/http-curl/client/HTTPClient.cpp:
##
@@ -378,10 +354,11 @@ int HTTPClient::onProgress(void *clientp, curl_off_t
/*dltotal*/, curl_off_t dln
return 0;
}
// did not transfer data
- if (elapsed.count() > client.read_timeout_ms_.count()) {
+ if (elapsed > client.read_timeout_) {
// timeout
-client.logger_->log_error("HTTP operation has been idle for %dms, limit
(%dms) reached, terminating connection\n",
- static_cast(elapsed.count()),
static_cast(client.read_timeout_ms_.count()));
+client.logger_->log_error("HTTP operation has been idle for %" PRId64 "
ms, limit (%" PRId64 "ms) reached, terminating connection\n",
+
std::chrono::duration_cast(elapsed).count(),
+ client.read_timeout_.count());
Review Comment:
good idea :+1:
https://github.com/apache/nifi-minifi-cpp/pull/1383/commits/3edf5fcfd603463b67bd6c072fa0128bbb861787#diff-e83ea756e38995b3194ecc99d0a64fc19c8ef992fb7a575a34def27a67f7819aR380-R381
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952696192
##
extensions/http-curl/client/HTTPClient.cpp:
##
@@ -32,62 +31,53 @@
#include "range/v3/algorithm/all_of.hpp"
#include "range/v3/action/transform.hpp"
-namespace org::apache::nifi::minifi::utils {
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::extensions::curl {
-HTTPClient::HTTPClient(std::string url, const
std::shared_ptr ssl_context_service)
+HTTPClient::HTTPClient(std::string url,
std::shared_ptr ssl_context_service)
: core::Connectable("HTTPClient"),
- ssl_context_service_(ssl_context_service),
+ ssl_context_service_(std::move(ssl_context_service)),
url_(std::move(url)) {
- http_session_ = curl_easy_init();
+ http_session_.reset(curl_easy_init());
}
HTTPClient::HTTPClient(const std::string& name, const utils::Identifier& uuid)
: core::Connectable(name, uuid) {
- http_session_ = curl_easy_init();
+ http_session_.reset(curl_easy_init());
}
HTTPClient::HTTPClient()
: core::Connectable("HTTPClient") {
- http_session_ = curl_easy_init();
+ http_session_.reset(curl_easy_init());
}
-void HTTPClient::addFormPart(const std::string& content_type, const
std::string& name, HTTPUploadCallback* read_callback, const
std::optional& filename) {
+void HTTPClient::addFormPart(const std::string& content_type, const
std::string& name, std::unique_ptr&& form_callback,
const std::optional& filename) {
Review Comment:
makes sense, I fixed these in
https://github.com/apache/nifi-minifi-cpp/pull/1383/commits/3edf5fcfd603463b67bd6c072fa0128bbb861787#diff-475912c726caf957fa46ce1b55d3e845e89aa19b4c7caf2cd4a0ae1f860cbdd5R98
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952695422
##
extensions/http-curl/client/HTTPCallback.h:
##
@@ -156,11 +152,11 @@ class HttpStreamingCallback final : public
ByteInputCallback {
current_pos_ = current_buffer_start_;
total_bytes_loaded_ += current_vec_.size();
logger_->log_trace("loadNextBuffer() loaded new buffer, ptr_: %p, size:
%zu, current_buffer_start_: %zu, current_pos_: %zu, total_bytes_loaded_: %zu",
- ptr_,
- current_vec_.size(),
- current_buffer_start_,
- current_pos_,
- total_bytes_loaded_);
+ ptr_,
+ current_vec_.size(),
+ current_buffer_start_,
+ current_pos_,
+ total_bytes_loaded_);
Review Comment:
fixed the indentation issues in
https://github.com/apache/nifi-minifi-cpp/pull/1383/commits/3edf5fcfd603463b67bd6c072fa0128bbb861787
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1383: MINIFICPP-1875 HTTPClient should be reusable
martinzink commented on code in PR #1383:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1383#discussion_r952694257
##
docker/test/integration/steps/steps.py:
##
@@ -271,6 +271,12 @@ def step_impl(context, content, path):
context.test.add_test_data(path, content)
+@given("{number_of_files:d} files with the content \"{content}\" are present
in \"{path}\"")
+def step_impl(context, number_of_files, content, path):
Review Comment:
makes sense, fixed it in
https://github.com/apache/nifi-minifi-cpp/pull/1383/commits/3edf5fcfd603463b67bd6c072fa0128bbb861787#diff-dce84c359f2f7128e501f5a322d8f6ac3325c2b471844f2c42cbef9abca58185R727
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #6317: NIFI-10375: If a class is not allowed in the AllowListClassLoader by …
markap14 commented on PR #6317: URL: https://github.com/apache/nifi/pull/6317#issuecomment-1224126078 Thanks for reviewing @turcsanyip . Pushed a fix that addresses both concerns. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10383) When importing flow that uses InvokeScriptedProcessor, if processor references service, referenced ID is incorrect
[ https://issues.apache.org/jira/browse/NIFI-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-10383: Fix Version/s: 1.18.0 Resolution: Fixed Status: Resolved (was: Patch Available) > When importing flow that uses InvokeScriptedProcessor, if processor > references service, referenced ID is incorrect > -- > > Key: NIFI-10383 > URL: https://issues.apache.org/jira/browse/NIFI-10383 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework, Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.18.0 > > Time Spent: 50m > Remaining Estimate: 0h > > When a flow is imported, if it contains an InvokeScriptedProcessor that uses > a Controller Service, the InvokeScriptedProcessor that is created references > the ID of the Controller Service when the flow was exported - NOT the ID of > the Controller Service that is created during flow import. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10383) When importing flow that uses InvokeScriptedProcessor, if processor references service, referenced ID is incorrect
[ https://issues.apache.org/jira/browse/NIFI-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583644#comment-17583644 ] ASF subversion and git services commented on NIFI-10383: Commit 9070cd0904e13bd1a992ebfdd6dddaabbb29ebe4 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9070cd0904 ] NIFI-10383: When importing/synching to a VersionedFlow, if the Versioned PropertyDescriptor indicates that a property references a Controller Service, treat it the same as if the PropertyDescriptor itself indicates that it references a service. This allows us to ensure that scripted components' property descriptors that reference controller services are properly mapped. Additionally updated StandardProcessorNode so that when we have this condition that we properly account for the Controller Service reference Signed-off-by: Matthew Burgess This closes #6322 > When importing flow that uses InvokeScriptedProcessor, if processor > references service, referenced ID is incorrect > -- > > Key: NIFI-10383 > URL: https://issues.apache.org/jira/browse/NIFI-10383 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework, Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > When a flow is imported, if it contains an InvokeScriptedProcessor that uses > a Controller Service, the InvokeScriptedProcessor that is created references > the ID of the Controller Service when the flow was exported - NOT the ID of > the Controller Service that is created during flow import. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 closed pull request #6322: NIFI-10383: When importing/synching to a VersionedFlow, if the Versio…
mattyb149 closed pull request #6322: NIFI-10383: When importing/synching to a VersionedFlow, if the Versio… URL: https://github.com/apache/nifi/pull/6322 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #6322: NIFI-10383: When importing/synching to a VersionedFlow, if the Versio…
mattyb149 commented on PR #6322: URL: https://github.com/apache/nifi/pull/6322#issuecomment-1224118902 +1 LGTM, tested a flow with an InvokeScriptedProcessor that references a DistributedMapCacheClientService, verified the reference was maintained. Thanks for the fix! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #6317: NIFI-10375: If a class is not allowed in the AllowListClassLoader by …
markap14 commented on code in PR #6317:
URL: https://github.com/apache/nifi/pull/6317#discussion_r952648095
##
nifi-stateless/nifi-stateless-bootstrap/src/main/java/org/apache/nifi/stateless/bootstrap/AllowListClassLoader.java:
##
@@ -37,36 +43,92 @@
*
*/
public class AllowListClassLoader extends ClassLoader {
-private final Set allowed;
+private static final Logger logger =
LoggerFactory.getLogger(AllowListClassLoader.class);
+
+private final Set allowedClassNames;
+private final List allowedModulePrefixes = Arrays.asList("java.",
"jdk.");
Review Comment:
Yeah, good point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10352) Remove an unused code for GenerateTableFetch.java
[ https://issues.apache.org/jira/browse/NIFI-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10352: Fix Version/s: (was: 1.18.0) > Remove an unused code for GenerateTableFetch.java > - > > Key: NIFI-10352 > URL: https://issues.apache.org/jira/browse/NIFI-10352 > Project: Apache NiFi > Issue Type: Improvement >Reporter: ZhangCheng >Assignee: ZhangCheng >Priority: Minor > Attachments: image-2022-08-13-20-13-23-493.png > > Time Spent: 10m > Remaining Estimate: 0h > > > remove the meaningless code of GenerateTableFetch.java, It is confusing for > reading code. > !image-2022-08-13-20-13-23-493.png|width=850,height=455! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (NIFI-10352) Remove an unused code for GenerateTableFetch.java
[
https://issues.apache.org/jira/browse/NIFI-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann reopened NIFI-10352:
-
[~Ku_Cheng] Please avoid marking an issue as {{Fixed}} until the pull request
is merged. Thanks!
> Remove an unused code for GenerateTableFetch.java
> -
>
> Key: NIFI-10352
> URL: https://issues.apache.org/jira/browse/NIFI-10352
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Minor
> Fix For: 1.18.0
>
> Attachments: image-2022-08-13-20-13-23-493.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> remove the meaningless code of GenerateTableFetch.java, It is confusing for
> reading code.
> !image-2022-08-13-20-13-23-493.png|width=850,height=455!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10120) Refactor CassandraSessionProvider to use the latest Cassandra driver
[
https://issues.apache.org/jira/browse/NIFI-10120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583613#comment-17583613
]
Steven Matison commented on NIFI-10120:
---
Options map now working for consistency level, socket options, and protocol
options. (/)
Options doc is here with allowed values:
[https://docs.datastax.com/en/drivers/java/4.7/com/datastax/oss/driver/api/core/config/TypedDriverOption.html]
Cleaning up some comments..
{code:java}
// the create function exists in following locations:
//
nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/AbstractCassandraProcessor.java
//
nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/AbstractCassandraProcessorTest.java
//
nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/PutCassandraQLTest.java
//
nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/PutCassandraRecordTest.java
//
nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/QueryCassandraTest.java
// new final 4.x function will need exist in all locations
{code}
latest updates here:
[https://github.com/steven-matison/nifi/commit/297e3e2f30639903d2a6d85c6361f0b547f71921]
> Refactor CassandraSessionProvider to use the latest Cassandra driver
>
>
> Key: NIFI-10120
> URL: https://issues.apache.org/jira/browse/NIFI-10120
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Mike Thomsen
>Assignee: Steven Matison
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
