[GitHub] [nifi] markobean commented on issue #3653: NIFI-6477 Delete operate policy when its component is deleted
markobean commented on issue #3653: NIFI-6477 Delete operate policy when its component is deleted URL: https://github.com/apache/nifi/pull/3653#issuecomment-522193178 +1 Ran build and installed the fix. Overrode 'operate the component' policy. Observed results in authorization.xml. Removed component. Observed policy removed in authorizations.xml. Works like a charm. Thanks Koji! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6564) Update/delete record from Bigquery
[ https://issues.apache.org/jira/browse/NIFI-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] thuy le updated NIFI-6564: -- Component/s: (was: Tools and Build) Extensions > Update/delete record from Bigquery > -- > > Key: NIFI-6564 > URL: https://issues.apache.org/jira/browse/NIFI-6564 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 >Reporter: thuy le >Priority: Major > > Update/delete record from Bigquery -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (NIFI-6564) Update/delete record from Bigquery
thuy le created NIFI-6564: - Summary: Update/delete record from Bigquery Key: NIFI-6564 URL: https://issues.apache.org/jira/browse/NIFI-6564 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Affects Versions: 1.9.2 Reporter: thuy le Update/delete record from Bigquery -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi] MikeThomsen commented on issue #3646: NIFI-6546 - Add JsonPath set value support
MikeThomsen commented on issue #3646: NIFI-6546 - Add JsonPath set value support URL: https://github.com/apache/nifi/pull/3646#issuecomment-522181957 I'll try to add this to my todo list. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] MikeThomsen commented on a change in pull request #3613: NIFI-6502 Add padLeft() and padRight() functions to RecordPath functions
MikeThomsen commented on a change in pull request #3613: NIFI-6502 Add
padLeft() and padRight() functions to RecordPath functions
URL: https://github.com/apache/nifi/pull/3613#discussion_r314919359
##
File path:
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java
##
@@ -78,25 +78,63 @@ public static String join(final Collection collection,
String delimiter) {
return sb.toString().substring(0, sb.lastIndexOf(delimiter));
}
-public static String padLeft(final String source, int length, char
padding) {
-if (source != null) {
+public static String padLeft(final String source, int length, String
padding) {
Review comment:
Did you write this yourself, or did you borrow it from somewhere like
`commons-lang3`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
[
https://issues.apache.org/jira/browse/NIFI-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909439#comment-16909439
]
Nathan Gough commented on NIFI-6561:
I found that this warning is not occurring every time Java is run. With some
testing I found that it also seemed to occur on JRE8 on the first boot after
switching JAVA_HOMEs. On restart, I found that the WARN did not occur the
second time NiFi was running. Then I couldn't get the warning to occur at all.
Seems a bit sporadic so I'll need to do more testing.
> Certificate compatibility broken for JDK8 build running on JRE11
>
>
> Key: NIFI-6561
> URL: https://issues.apache.org/jira/browse/NIFI-6561
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Security
>Affects Versions: 1.10.0
>Reporter: Nathan Gough
>Priority: Major
> Labels: Java11, certificate, tls
>
> When testing Java 11 build compatibility, I found an issue with TLS
> certificates when using a remote process group looped back to an input port
> on the same cluster. The same certificates were used for JDK8/JRE8,
> JDK8/JRE11, JDK11/JRE11 ie. they contained relevant SAN entries in each case.
> *Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
> attempting to send to local input port with RPG*:
> {code:java}
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> natog0.com:9551 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
> doesn't match any of the subject alternative names: [natog1.com]
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster{code}
> But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [nifi-minifi-cpp] asfgit closed pull request #595: Add windows tests
asfgit closed pull request #595: Add windows tests URL: https://github.com/apache/nifi-minifi-cpp/pull/595 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] mcgilman commented on issue #3602: NIFI-5839 Applied identity mapping to user lookups and group members
mcgilman commented on issue #3602: NIFI-5839 Applied identity mapping to user lookups and group members URL: https://github.com/apache/nifi/pull/3602#issuecomment-522155731 @dtmo Thanks for creating this PR! I recently ran into the same problem and was going to post a PR (it's not quite ready yet) with a slightly different solution when I came across this. I took a slightly different approach and wanted to discuss here. While we could leverage the user/group mappings to transform the values for this use case within the `LdapUserGroupProvider`, the mappings are meant to be applied to the identities and names that come out of a given (any) provider and the identity of the user upon authentication. The comparison happening in this scenario is happening within the `LdapUserGroupProvider` only. The value that associates a user with a group and/or a group with a user comes from an attribute in the group or user respectively. The directory server will not be performing any mappings to associate these two entries. The reason why this is problematic today is that the directory servers may or may not enforce case. I would like to suggest that we do not use the mappings to transform the values to support this scenario. If a user did not care to map the user identities or group names, but the did have this problem, they would need to create mapping entries in `nifi.properties` just to support their `LdapUserGroupProvider` configuration. This could potentially affect other providers (if configured to possibly use a composite provider) and when users authenticate. I would like to consider introducing a new property for the `LdapUserGroupProvider` that can conditionally set whether group membership decisions are case sensitive or not. This should hopefully lessen the already confusion configuration and limit the potential effects of this change. I should have a PR ready for consideration soon. When I do, I'll link it here. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless
init from flow xml
URL: https://github.com/apache/nifi/pull/3651#discussion_r314430591
##
File path:
nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/ComponentFactory.java
##
@@ -223,4 +350,24 @@ private Bundle getAvailableBundle(final
org.apache.nifi.registry.flow.Bundle bun
return possibleBundles.get(0);
}
+
+private Bundle getAvailableBundle(final BundleDTO bundle, final String
componentType) {
Review comment:
fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml URL: https://github.com/apache/nifi/pull/3651#discussion_r314430500 ## File path: nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/ComponentFactory.java ## @@ -204,6 +281,56 @@ private ControllerService createControllerService(final VersionedControllerServi } } +private ControllerService createControllerService(final ControllerServiceDTO controllerService, final VariableRegistry variableRegistry, final Set classpathUrls, Review comment: fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless
init from flow xml
URL: https://github.com/apache/nifi/pull/3651#discussion_r314891025
##
File path:
nifi-stateless/nifi-stateless-core/src/test/java/org/apache/nifi/stateless/core/InitFromFlowXmlIT.java
##
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stateless.core;
+
+
+//import com.google.gson.JsonObject;
+//import com.google.gson.JsonParser;
+//import org.apache.nifi.bundle.Bundle;
+//import org.apache.nifi.bundle.BundleCoordinate;
+//import org.apache.nifi.controller.exception.ProcessorInstantiationException;
+//import org.apache.nifi.controller.serialization.FlowEncodingVersion;
+//import org.apache.nifi.controller.serialization.FlowFromDOMFactory;
+//import org.apache.nifi.encrypt.StringEncryptor;
+//import org.apache.nifi.nar.ExtensionManager;
+//import org.apache.nifi.nar.NarUnpacker;
+//import org.apache.nifi.parameter.Parameter;
+//import org.apache.nifi.parameter.ParameterContext;
+//import org.apache.nifi.registry.VariableDescriptor;
+//import org.apache.nifi.reporting.InitializationException;
+//import org.apache.nifi.stateless.bootstrap.ExtensionDiscovery;
+//import org.apache.nifi.util.NiFiProperties;
+//import org.apache.nifi.web.api.dto.ProcessGroupDTO;
+//import org.glassfish.jersey.internal.jsr166.Flow;
+//import org.w3c.dom.Document;
+//import org.w3c.dom.Element;
+//
+//import javax.net.ssl.SSLContext;
+//import java.io.File;
+//import java.io.FileNotFoundException;
+//import java.io.IOException;
+//import java.net.MalformedURLException;
+//import java.net.URL;
+//import java.net.URLClassLoader;
+//import java.nio.file.Path;
+//import java.nio.file.Paths;
+//import java.util.*;
+//
+//import static org.mockito.ArgumentMatchers.anyString;
+//import static org.mockito.Mockito.mock;
+//import static org.mockito.Mockito.when;
+
+public class InitFromFlowXmlIT {
+
+//private NiFiProperties getNiFiProperties() {
Review comment:
This was failing when trying to load the processor classes from the standard
processors nar. I think it has to do with the narloader but I wasn't sure where
to go from there. I'll remove this and add unit tests for the flowxml parser
instead
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless
init from flow xml
URL: https://github.com/apache/nifi/pull/3651#discussion_r314426530
##
File path:
nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/ComponentFactory.java
##
@@ -120,6 +124,74 @@ public StatelessProcessorWrapper createProcessor(final
VersionedProcessor versio
}
}
+public StatelessProcessorWrapper createProcessor(final ProcessorDTO
processorDto, final boolean materializeContent, final
StatelessControllerServiceLookup controllerServiceLookup,
+ final VariableRegistry
variableRegistry, final Set classpathUrls, final ParameterContext
parameterContext)
+throws ProcessorInstantiationException {
+
+final String type = processorDto.getType();
+final String identifier = processorDto.getId();
+
+final Bundle bundle = getAvailableBundle(processorDto.getBundle(),
type);
+if (bundle == null) {
+throw new IllegalStateException("Unable to find bundle for
coordinate "
++ processorDto.getBundle().getGroup() + ":"
++ processorDto.getBundle().getArtifact() + ":"
++ processorDto.getBundle().getVersion());
+}
+
+final ClassLoader ctxClassLoader =
Thread.currentThread().getContextClassLoader();
+try {
+final ClassLoader detectedClassLoader =
extensionManager.createInstanceClassLoader(type, identifier, bundle,
+classpathUrls == null ? Collections.emptySet() :
classpathUrls);
+
+logger.debug("Setting context class loader to {} (parent = {}) to
create {}", detectedClassLoader, detectedClassLoader.getParent(), type);
+final Class rawClass = Class.forName(type, true,
detectedClassLoader);
+Thread.currentThread().setContextClassLoader(detectedClassLoader);
+
+final Object extensionInstance = rawClass.newInstance();
+final ComponentLog componentLog = new
SLF4JComponentLog(extensionInstance);
+
+final Processor processor = (Processor) extensionInstance;
+final ProcessorInitializationContext initializationContext = new
StatelessProcessorInitializationContext(processor.getIdentifier(), processor,
controllerServiceLookup);
Review comment:
yes, good catch
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless
init from flow xml
URL: https://github.com/apache/nifi/pull/3651#discussion_r314890195
##
File path:
nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/StatelessFlow.java
##
@@ -99,6 +122,194 @@ public StatelessFlow(final List
roots) {
this.componentFactory = null;
}
+public StatelessFlow(final ProcessGroupDTO flow, final ExtensionManager
extensionManager, final VariableRegistry variableRegistry, final List
failureOutputPorts,
Review comment:
I'm not sure how I should approach this one. I started with using the
strategy pattern to get around the sleight differences between `Versioned*`
objects and `*DTO` objects but I feel like I ended up with too many generics
for what I'm trying to achieve. Do you have any recommendations for how to
refactor this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml URL: https://github.com/apache/nifi/pull/3651#discussion_r314430423 ## File path: nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/ComponentFactory.java ## @@ -120,6 +124,74 @@ public StatelessProcessorWrapper createProcessor(final VersionedProcessor versio } } +public StatelessProcessorWrapper createProcessor(final ProcessorDTO processorDto, final boolean materializeContent, final StatelessControllerServiceLookup controllerServiceLookup, Review comment: Totally agree about all the code duplication. I was wary about it when I submitted the PR but this is my first contribution to NiFi and I didn't know what the appetite would be for significant code changes to an existing feature. And tbh I'm not much of a java guy anyway so thanks for the pointers on how to clean this up. I've brought in your suggestions, the only thing I modified is to remove the generic for the private methods as I don't think they're needed since we already extracted the params we needed and intellij was complaining This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless
init from flow xml
URL: https://github.com/apache/nifi/pull/3651#discussion_r314473017
##
File path:
nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/StatelessRemoteOutputPort.java
##
@@ -84,6 +87,41 @@ public StatelessRemoteOutputPort(final
VersionedRemoteProcessGroup rpg, final Ve
.build();
}
+public StatelessRemoteOutputPort(final RemoteProcessGroupDTO rpg, final
RemoteProcessGroupPortDTO remotePort, final SSLContext sslContext) {
Review comment:
Changed to single constructor and moved the initialization code to
ComponentFactory
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless init from flow xml
dbkegley commented on a change in pull request #3651: NIFI-6539 Nifi stateless
init from flow xml
URL: https://github.com/apache/nifi/pull/3651#discussion_r314433073
##
File path:
nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/runtimes/Program.java
##
@@ -154,23 +161,27 @@ private static void runLocal(final String[] args, final
ClassLoader systemClassL
private static void printUsage() {
System.out.println("Usage:");
-System.out.println(" " + RUN_FROM_REGISTRY + " [Once|Continuous]
--json ");
+System.out.println("1) " + RUN_FROM_REGISTRY + " [Once|Continuous]
--json ");
System.out.println(" " + RUN_FROM_REGISTRY + " [Once|Continuous]
--file ");
System.out.println();
-System.out.println(" " + RUN_YARN_SERVICE_FROM_REGISTRY + "
<# of Containers> --json
");
+System.out.println("2) " + RUN_FROM_FLOW_XML + " [Once|Continuous]
--json ");
+System.out.println(" " + RUN_FROM_FLOW_XML + " [Once|Continuous]
--file ");
+System.out.println();
+System.out.println("3) " + RUN_YARN_SERVICE_FROM_REGISTRY + "
<# of Containers> --json
");
System.out.println(" " + RUN_YARN_SERVICE_FROM_REGISTRY + "
<# of Containers> --file
");
System.out.println();
-System.out.println("3) " + RUN_OPENWHISK_ACTION_SERVER + "
");
+System.out.println("4) " + RUN_OPENWHISK_ACTION_SERVER + "
");
System.out.println();
System.out.println("Examples:");
System.out.println("1) " + RUN_FROM_REGISTRY + " Once --json
\"{\\\"registryUrl\\\":\\\"http://172.26.198.107:61080\\\",\\\"bucketId\\\":\\\"5eec8794-01b3-4cd7-8536-0167c8b4ce8c\\\",\\\"flowId\\\":
\\\"c5fa1d4f-b453-4bf5-8ff3-352352c418f3\\\"}\"");
-System.out.println("2) " + RUN_YARN_SERVICE_FROM_REGISTRY + "
http://127.0.0.1:8088 nifi-stateless:latest kafka-to-solr 3 --file
kafka-to-solr.json");
-System.out.println("3) " + RUN_OPENWHISK_ACTION_SERVER + " 8080");
+System.out.println("2) " + RUN_FROM_FLOW_XML + " Once --file
/opt/nifi/nifi-stateless-configs/flow-abc.json");
Review comment:
updated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (NIFI-3645) Add aws:kms, sse-c, sse-c-key, and sse-kms-key-id capability to server side encryption in the PutS3Object processor.
[ https://issues.apache.org/jira/browse/NIFI-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Troy Melhase resolved NIFI-3645. Resolution: Implemented > Add aws:kms, sse-c, sse-c-key, and sse-kms-key-id capability to server side > encryption in the PutS3Object processor. > > > Key: NIFI-3645 > URL: https://issues.apache.org/jira/browse/NIFI-3645 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Matthew Clarke >Priority: Major > > Server Side Encryption (sse) currently exists in the PutS3Object processor > but don’t see sse-c, sse-c-key or sse-kms-key-id options. > --sse (string) Specifies server-side encryption of the object in S3. Valid > values are AES256 and aws:kms. If the parameter is specified but no value is > provided, AES256 is used. > --sse-c (string) Specifies server-side encryption using customer provided > keys of the the object in S3. AES256 is the only valid value. If the > parameter is specified but no value is provided, AES256 is used. If you > provide this value, --sse-c-key must be specified as well. > --sse-c-key (string) The customer-provided encryption key to use to > server-side encrypt the object in S3. If you provide this value, --sse-c must > be specified as well. The key provided should not be base64 encoded. > --sse-kms-key-id (string) The AWS KMS key ID that should be used to > server-side encrypt the object in S3. Note that you should only provide this > parameter if KMS key ID is different the default S3 master KMS key. > Would like to see full support of the various server side encryption > capabilities added to our S3 processors. > This is related partially to another Apache Jira: NIFI-1769 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (NIFI-6382) Ensure proper integration between NiFi & NiFi Registry with respect to Parameters
[ https://issues.apache.org/jira/browse/NIFI-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-6382: -- Resolution: Fixed Fix Version/s: 1.10.0 Status: Resolved (was: Patch Available) > Ensure proper integration between NiFi & NiFi Registry with respect to > Parameters > - > > Key: NIFI-6382 > URL: https://issues.apache.org/jira/browse/NIFI-6382 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.10.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Need to ensure that flows saved to the Flow Registry have all Parameters also > passed in, with the values (except for sensitive parameters, which should be > defined but have no values). Need to ensure that when importing a flow, that > the Parameter Context is able to be chosen, or a new one created. If a new > one is created, it should be created with the values from registry. Also, > need to ensure that when changing version that any Parameters in the 'new' > version are also included in the Parameter Context (assuming adequate > permissions to add them). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (NIFI-6382) Ensure proper integration between NiFi & NiFi Registry with respect to Parameters
[ https://issues.apache.org/jira/browse/NIFI-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909382#comment-16909382 ] ASF subversion and git services commented on NIFI-6382: --- Commit bcf373a049dd8d393b8cc3c36e2a26950d07a636 in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bcf373a ] NIFI-6382: Allow Parameters to have null values Signed-off-by: Bryan Bende > Ensure proper integration between NiFi & NiFi Registry with respect to > Parameters > - > > Key: NIFI-6382 > URL: https://issues.apache.org/jira/browse/NIFI-6382 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Need to ensure that flows saved to the Flow Registry have all Parameters also > passed in, with the values (except for sensitive parameters, which should be > defined but have no values). Need to ensure that when importing a flow, that > the Parameter Context is able to be chosen, or a new one created. If a new > one is created, it should be created with the values from registry. Also, > need to ensure that when changing version that any Parameters in the 'new' > version are also included in the Parameter Context (assuming adequate > permissions to add them). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (NIFI-6382) Ensure proper integration between NiFi & NiFi Registry with respect to Parameters
[ https://issues.apache.org/jira/browse/NIFI-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909385#comment-16909385 ] ASF subversion and git services commented on NIFI-6382: --- Commit bd8342c5e06e71846d1e5976bfe9de45690b974d in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bd8342c ] NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary NIFI-6382: Fixed bugs reported in code review: sensitive properties referencing parameters should be exported to registry; when importing a versioned flow, any new parameters should be added to the parameter context NIFI-6382: Updated logic for handling parameter references with sensitive properties when integrating with nifi registry NIFI-6382: If user does not have permissions to READ a parameter context with a given name when importing a flow or changing flow version, ensure that NiFi properly generates an AccessDenied messages instead of throwing NullPointerException This closes #3642. Signed-off-by: Bryan Bende > Ensure proper integration between NiFi & NiFi Registry with respect to > Parameters > - > > Key: NIFI-6382 > URL: https://issues.apache.org/jira/browse/NIFI-6382 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Need to ensure that flows saved to the Flow Registry have all Parameters also > passed in, with the values (except for sensitive parameters, which should be > defined but have no values). Need to ensure that when importing a flow, that > the Parameter Context is able to be chosen, or a new one created. If a new > one is created, it should be created with the values from registry. Also, > need to ensure that when changing version that any Parameters in the 'new' > version are also included in the Parameter Context (assuming adequate > permissions to add them). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (NIFI-6382) Ensure proper integration between NiFi & NiFi Registry with respect to Parameters
[ https://issues.apache.org/jira/browse/NIFI-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909383#comment-16909383 ] ASF subversion and git services commented on NIFI-6382: --- Commit bd8342c5e06e71846d1e5976bfe9de45690b974d in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bd8342c ] NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary NIFI-6382: Fixed bugs reported in code review: sensitive properties referencing parameters should be exported to registry; when importing a versioned flow, any new parameters should be added to the parameter context NIFI-6382: Updated logic for handling parameter references with sensitive properties when integrating with nifi registry NIFI-6382: If user does not have permissions to READ a parameter context with a given name when importing a flow or changing flow version, ensure that NiFi properly generates an AccessDenied messages instead of throwing NullPointerException This closes #3642. Signed-off-by: Bryan Bende > Ensure proper integration between NiFi & NiFi Registry with respect to > Parameters > - > > Key: NIFI-6382 > URL: https://issues.apache.org/jira/browse/NIFI-6382 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Need to ensure that flows saved to the Flow Registry have all Parameters also > passed in, with the values (except for sensitive parameters, which should be > defined but have no values). Need to ensure that when importing a flow, that > the Parameter Context is able to be chosen, or a new one created. If a new > one is created, it should be created with the values from registry. Also, > need to ensure that when changing version that any Parameters in the 'new' > version are also included in the Parameter Context (assuming adequate > permissions to add them). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (NIFI-6382) Ensure proper integration between NiFi & NiFi Registry with respect to Parameters
[ https://issues.apache.org/jira/browse/NIFI-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909386#comment-16909386 ] ASF subversion and git services commented on NIFI-6382: --- Commit bd8342c5e06e71846d1e5976bfe9de45690b974d in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bd8342c ] NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary NIFI-6382: Fixed bugs reported in code review: sensitive properties referencing parameters should be exported to registry; when importing a versioned flow, any new parameters should be added to the parameter context NIFI-6382: Updated logic for handling parameter references with sensitive properties when integrating with nifi registry NIFI-6382: If user does not have permissions to READ a parameter context with a given name when importing a flow or changing flow version, ensure that NiFi properly generates an AccessDenied messages instead of throwing NullPointerException This closes #3642. Signed-off-by: Bryan Bende > Ensure proper integration between NiFi & NiFi Registry with respect to > Parameters > - > > Key: NIFI-6382 > URL: https://issues.apache.org/jira/browse/NIFI-6382 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Need to ensure that flows saved to the Flow Registry have all Parameters also > passed in, with the values (except for sensitive parameters, which should be > defined but have no values). Need to ensure that when importing a flow, that > the Parameter Context is able to be chosen, or a new one created. If a new > one is created, it should be created with the values from registry. Also, > need to ensure that when changing version that any Parameters in the 'new' > version are also included in the Parameter Context (assuming adequate > permissions to add them). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (NIFI-6382) Ensure proper integration between NiFi & NiFi Registry with respect to Parameters
[ https://issues.apache.org/jira/browse/NIFI-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909384#comment-16909384 ] ASF subversion and git services commented on NIFI-6382: --- Commit bd8342c5e06e71846d1e5976bfe9de45690b974d in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bd8342c ] NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary NIFI-6382: Fixed bugs reported in code review: sensitive properties referencing parameters should be exported to registry; when importing a versioned flow, any new parameters should be added to the parameter context NIFI-6382: Updated logic for handling parameter references with sensitive properties when integrating with nifi registry NIFI-6382: If user does not have permissions to READ a parameter context with a given name when importing a flow or changing flow version, ensure that NiFi properly generates an AccessDenied messages instead of throwing NullPointerException This closes #3642. Signed-off-by: Bryan Bende > Ensure proper integration between NiFi & NiFi Registry with respect to > Parameters > - > > Key: NIFI-6382 > URL: https://issues.apache.org/jira/browse/NIFI-6382 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Need to ensure that flows saved to the Flow Registry have all Parameters also > passed in, with the values (except for sensitive parameters, which should be > defined but have no values). Need to ensure that when importing a flow, that > the Parameter Context is able to be chosen, or a new one created. If a new > one is created, it should be created with the values from registry. Also, > need to ensure that when changing version that any Parameters in the 'new' > version are also included in the Parameter Context (assuming adequate > permissions to add them). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi] asfgit closed pull request #3642: NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary
asfgit closed pull request #3642: NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary URL: https://github.com/apache/nifi/pull/3642 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6563) Add support for TLSv1.3 when running on Java 11
[ https://issues.apache.org/jira/browse/NIFI-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated NIFI-6563: -- Issue Type: Sub-task (was: Improvement) Parent: NIFI-5174 > Add support for TLSv1.3 when running on Java 11 > --- > > Key: NIFI-6563 > URL: https://issues.apache.org/jira/browse/NIFI-6563 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework, Tools and Build >Affects Versions: 1.9.2 >Reporter: Jeff Storck >Priority: Major > > Add full support for TLSv1.3 running on Java 11. > Some components currently work (S2S over HTTP, for example) with TLSv1.3. > One issue related to specifying "TLS" for the protocol in an SSLContext is > different (implied) behavior between running on Java 8 and 11, since the > "newest" version of the TLS protocol available on the JVM will be used. > More details will be added upon further analysis. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (NIFI-6563) Add support for TLSv1.3 when running on Java 11
Jeff Storck created NIFI-6563: - Summary: Add support for TLSv1.3 when running on Java 11 Key: NIFI-6563 URL: https://issues.apache.org/jira/browse/NIFI-6563 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Tools and Build Affects Versions: 1.9.2 Reporter: Jeff Storck Add full support for TLSv1.3 running on Java 11. Some components currently work (S2S over HTTP, for example) with TLSv1.3. One issue related to specifying "TLS" for the protocol in an SSLContext is different (implied) behavior between running on Java 8 and 11, since the "newest" version of the TLS protocol available on the JVM will be used. More details will be added upon further analysis. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi] arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP
SiteToSite server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314882004
##
File path:
nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/http/HttpClient.java
##
@@ -168,6 +173,10 @@ public Transaction createTransaction(final
TransferDirection direction) throws H
try {
transactionUrl = apiClient.initiateTransaction(direction,
portId);
commSession.setUserDn(apiClient.getTrustedPeerDn());
+} catch (final NoContentException e) {
Review comment:
Good spot, I think we should, will add.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP
SiteToSite server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314881854
##
File path:
nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java
##
@@ -1224,6 +1249,24 @@ public void testReceiveUnknownPort() throws Exception {
}
}
+@Test
+public void testReceiveEmptyPort() throws Exception {
+
+try (
+SiteToSiteClient client = getDefaultBuilder()
+.portName("empty-output-running")
+.build()
+) {
+try {
+client.createTransaction(TransferDirection.RECEIVE);
+fail();
+} catch (NoContentException e) {
+logger.info("Exception message: {}", e.getMessage());
+assertTrue(e.getMessage().contains("Server has no flowfiles to
provide"));
Review comment:
According to the discussion above between @ijokarumawak and me, this TC will
most probably change anyway.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] bbende commented on issue #3642: NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary
bbende commented on issue #3642: NIFI-6382: On flow import and version change, create parameter context where necessary and add parameters where necessary URL: https://github.com/apache/nifi/pull/3642#issuecomment-522143135 Verified the latest changes, everything looks good, going to merge, thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP
SiteToSite server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314880039
##
File path:
nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/http/HttpClient.java
##
@@ -168,6 +169,9 @@ public Transaction createTransaction(final
TransferDirection direction) throws H
try {
transactionUrl = apiClient.initiateTransaction(direction,
portId);
commSession.setUserDn(apiClient.getTrustedPeerDn());
+} catch (final NoContentException e) {
+logger.debug("Peer {} has no flowfiles to provide", peer);
+throw e;
Review comment:
Generally I like the idea, the impact is going to be way bigger than the
Jira itself would justify, but I'm fine with that :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP
SiteToSite server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314880039
##
File path:
nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/http/HttpClient.java
##
@@ -168,6 +169,9 @@ public Transaction createTransaction(final
TransferDirection direction) throws H
try {
transactionUrl = apiClient.initiateTransaction(direction,
portId);
commSession.setUserDn(apiClient.getTrustedPeerDn());
+} catch (final NoContentException e) {
+logger.debug("Peer {} has no flowfiles to provide", peer);
+throw e;
Review comment:
Generally I like the idea, the impact is going to be way more than the Jira
itself would justify, but I'm fine with that :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP
SiteToSite server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314878172
##
File path:
nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/http/HttpClient.java
##
@@ -168,6 +173,10 @@ public Transaction createTransaction(final
TransferDirection direction) throws H
try {
transactionUrl = apiClient.initiateTransaction(direction,
portId);
commSession.setUserDn(apiClient.getTrustedPeerDn());
+} catch (final NoContentException e) {
Review comment:
I think we should, thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP
SiteToSite server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314876822
##
File path:
nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/StatelessRemoteOutputPort.java
##
@@ -139,7 +140,11 @@ public boolean runRecursive(final Queue
queue) {
transaction.confirm();
transaction.complete();
-} catch (final Exception e) {
+} catch (final NoContentException e) {
+getLogger().info("No flowfiles to receive");
Review comment:
Good spot, will change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
arpadboda commented on a change in pull request #3647: NIFI-6530 - HTTP
SiteToSite server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314876307
##
File path:
nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java
##
@@ -1224,6 +1249,24 @@ public void testReceiveUnknownPort() throws Exception {
}
}
+@Test
+public void testReceiveEmptyPort() throws Exception {
Review comment:
We definitely don't want to penalize peer in case of no content.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-registry] markap14 opened a new pull request #219: NIFI-309: Added Difference Types PROPERTY_PARAMETERIZED and PROPERTY_…
markap14 opened a new pull request #219: NIFI-309: Added Difference Types PROPERTY_PARAMETERIZED and PROPERTY_… URL: https://github.com/apache/nifi-registry/pull/219 …PARAMETERIZATION_REMOVED This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFIREG-309) Flow difference descriptions for sensitive properties referencing parameters are unclear
Mark Payne created NIFIREG-309: -- Summary: Flow difference descriptions for sensitive properties referencing parameters are unclear Key: NIFIREG-309 URL: https://issues.apache.org/jira/browse/NIFIREG-309 Project: NiFi Registry Issue Type: Improvement Affects Versions: 0.5.0 Reporter: Mark Payne Assignee: Mark Payne With the addition of Parameters, we can now store a sensitive property in the NiFi Registry if the property value is simply a parameter reference. Otherwise the value is still blanked out. However, when a Flow Comparison is done, if a sensitive property goes from being a static value to a parameterized value, the Flow Difference is shown as PROPERTY_ADDED because the value was previously nulled out and non longer is. Likewise, going from a parameterized value to an explicit value shows as a PROPERTY_REMOVED Difference Type, even though the property was not actually removed, it was just changed from a Parameter Reference to an explicit value. We should update the Flow Comparitor to have two new Difference Types, one for a property being parameterized and one for a property be un-parameterized. We also will need to update the DifferenceDescriptor's to make the wording clear. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi] adarmiento commented on issue #3631: NIFI-6513 PutHive3Streaming.AUTO_CREATE_PARTITIONS is not used
adarmiento commented on issue #3631: NIFI-6513 PutHive3Streaming.AUTO_CREATE_PARTITIONS is not used URL: https://github.com/apache/nifi/pull/3631#issuecomment-522118071 Hello @mattyb149 Do you think the additional doc is enough? Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-6562) Expose ExecutionNode in ProcessContext
Peter Wicks created NIFI-6562: - Summary: Expose ExecutionNode in ProcessContext Key: NIFI-6562 URL: https://issues.apache.org/jira/browse/NIFI-6562 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Wicks Assignee: Peter Wicks In some circumstances it can be helpful to know if a processor is schedule to run on All nodes or only the Primary node, and take appropriate action when the primary node changes based on this context. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (NIFI-5952) RAW Site-to-Site fails with java.nio.channels.IllegalBlockingModeException
[
https://issues.apache.org/jira/browse/NIFI-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Storck updated NIFI-5952:
--
Labels: Java11 (was: )
> RAW Site-to-Site fails with java.nio.channels.IllegalBlockingModeException
> --
>
> Key: NIFI-5952
> URL: https://issues.apache.org/jira/browse/NIFI-5952
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core Framework
> Environment: jdk-11.0.1
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Critical
> Labels: Java11
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> During the review cycle of NIFI-5820, I found that while HTTP S2S works
> without issue, RAW S2S is failing with following Exception:
> {code:java}
> 2018-12-19 16:19:26,811 ERROR [Site-to-Site Listener] org.apache.nifi.NiFi
> java.nio.channels.IllegalBlockingModeException: null
> at
> java.base/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:121)
> at
> org.apache.nifi.remote.SocketRemoteSiteListener$1.run(SocketRemoteSiteListener.java:125)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> Despite of the fact that the RAW has been worked with older Java versions, it
> seems current nio usage at RAW S2S is not correct. And JDK 11 starts
> complaining about it.
> Here are few things I've discovered with current NiFi and nio SocketChannel:
> - NiFi accepts RAW S2S client connection with SocketRemoteSiteListener,
> which uses ServerSocketChannel as non-blocking manner [1]
> - But SocketRemoteSiteListener doesn't use Selector API to accept incoming
> connection and transfer data with the channel. This is the cause of above
> exception.
> - SocketRemoteSiteListener spawns new thread when it accepts connection.
> This is how connections are handled with a non-nio, standard Socket
> programming. If we want to use non-blocking NIO, we need to use channels with
> Selector
> - But using non-blocking IO with current NiFi S2S protocol can only add few
> or none benefit by doing so. [2]
> To make RAW S2S work with Java 11, we need either:
> A. Stop using nio packages.
> B. Implement correct nio usage, meaning use Selector IO and probably we need
> another thread pool.
> I'm going to take the approach A above, because B would take much more
> refactoring.
> [1]
> [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/SocketRemoteSiteListener.java#L120]
> [2]
> [https://stackoverflow.com/questions/12338204/in-java-nio-is-a-selector-useful-for-a-client-socketchannel]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
[
https://issues.apache.org/jira/browse/NIFI-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Storck updated NIFI-6561:
--
Issue Type: Sub-task (was: Improvement)
Parent: NIFI-5174
> Certificate compatibility broken for JDK8 build running on JRE11
>
>
> Key: NIFI-6561
> URL: https://issues.apache.org/jira/browse/NIFI-6561
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Security
>Affects Versions: 1.10.0
>Reporter: Nathan Gough
>Priority: Major
> Labels: Java11, certificate, tls
>
> When testing Java 11 build compatibility, I found an issue with TLS
> certificates when using a remote process group looped back to an input port
> on the same cluster. The same certificates were used for JDK8/JRE8,
> JDK8/JRE11, JDK11/JRE11 ie. they contained relevant SAN entries in each case.
> *Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
> attempting to send to local input port with RPG*:
> {code:java}
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> natog0.com:9551 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
> doesn't match any of the subject alternative names: [natog1.com]
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster{code}
> But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
[
https://issues.apache.org/jira/browse/NIFI-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Storck updated NIFI-6561:
--
Labels: Java11 certificate tls (was: certificate tls)
> Certificate compatibility broken for JDK8 build running on JRE11
>
>
> Key: NIFI-6561
> URL: https://issues.apache.org/jira/browse/NIFI-6561
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Security
>Affects Versions: 1.10.0
>Reporter: Nathan Gough
>Priority: Major
> Labels: Java11, certificate, tls
>
> When testing Java 11 build compatibility, I found an issue with TLS
> certificates when using a remote process group looped back to an input port
> on the same cluster. The same certificates were used for JDK8/JRE8,
> JDK8/JRE11, JDK11/JRE11 ie. they contained relevant SAN entries in each case.
> *Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
> attempting to send to local input port with RPG*:
> {code:java}
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> natog0.com:9551 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
> doesn't match any of the subject alternative names: [natog1.com]
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster{code}
> But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
[
https://issues.apache.org/jira/browse/NIFI-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-6561:
---
Description:
When testing Java 11 build compatibility, I found an issue with TLS
certificates when using a remote process group looped back to an input port on
the same cluster. The same certificates were used for JDK8/JRE8, JDK8/JRE11,
JDK11/JRE11 ie. they contained relevant SAN entries in each case.
*Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
attempting to send to local input port with RPG*:
{noformat}
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector Could not communicate with
natog0.com:9551 to determine which nodes exist in the remote NiFi cluster, due
to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
doesn't match any of the subject alternative names: [natog1.com]
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
Group's peers due to Unable to communicate with remote NiFi cluster in order to
determine which nodes exist in the remote cluster{noformat}
But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
was:
When testing Java 11 build compatibility, I found an issue with TLS
certificates when using a remote process group looped back to an input port on
the same cluster. The same certificates were used for JDK8/JRE8, JDK8/JRE11,
JDK11/JRE11 ie. they contained relevant SAN entries in each case.
*Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
attempting to send to local input port with RPG*:
{code:java}
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector Could not communicate with
natog0.com:9551 to determine which nodes exist in the remote NiFi cluster, due
to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
doesn't match any of the subject alternative names: [natog1.com] 2019-08-13
18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
Group's peers due to Unable to communicate with remote NiFi cluster in order to
determine which nodes exist in the remote cluster{code}
But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
> Certificate compatibility broken for JDK8 build running on JRE11
>
>
> Key: NIFI-6561
> URL: https://issues.apache.org/jira/browse/NIFI-6561
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Security
>Affects Versions: 1.10.0
>Reporter: Nathan Gough
>Priority: Major
> Labels: certificate, tls
>
> When testing Java 11 build compatibility, I found an issue with TLS
> certificates when using a remote process group looped back to an input port
> on the same cluster. The same certificates were used for JDK8/JRE8,
> JDK8/JRE11, JDK11/JRE11 ie. they contained relevant SAN entries in each case.
> *Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
> attempting to send to local input port with RPG*:
> {noformat}
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> natog0.com:9551 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
> doesn't match any of the subject alternative names: [natog1.com]
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster{noformat}
> But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
[
https://issues.apache.org/jira/browse/NIFI-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-6561:
---
Description:
When testing Java 11 build compatibility, I found an issue with TLS
certificates when using a remote process group looped back to an input port on
the same cluster. The same certificates were used for JDK8/JRE8, JDK8/JRE11,
JDK11/JRE11 ie. they contained relevant SAN entries in each case.
*Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
attempting to send to local input port with RPG*:
{code:java}
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector Could not communicate with
natog0.com:9551 to determine which nodes exist in the remote NiFi cluster, due
to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
doesn't match any of the subject alternative names: [natog1.com] 2019-08-13
18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
Group's peers due to Unable to communicate with remote NiFi cluster in order to
determine which nodes exist in the remote cluster{code}
But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
was:
When testing Java 11 build compatibility, I found an issue with TLS
certificates when using a remote process group looped back to an input port on
the same cluster. The same certificates were used for JDK8/JRE8, JDK8/JRE11,
JDK11/JRE11 ie. they contained relevant SAN entries in each case.
*Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
attempting to send to local input port with RPG*:
{{}}
{code:java}
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector Could not communicate with
natog0.com:9551 to determine which nodes exist in the remote NiFi cluster, due
to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
doesn't match any of the subject alternative names: [natog1.com] 2019-08-13
18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
Group's peers due to Unable to communicate with remote NiFi cluster in order to
determine which nodes exist in the remote cluster{code}
But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
> Certificate compatibility broken for JDK8 build running on JRE11
>
>
> Key: NIFI-6561
> URL: https://issues.apache.org/jira/browse/NIFI-6561
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Security
>Affects Versions: 1.10.0
>Reporter: Nathan Gough
>Priority: Major
> Labels: certificate, tls
>
> When testing Java 11 build compatibility, I found an issue with TLS
> certificates when using a remote process group looped back to an input port
> on the same cluster. The same certificates were used for JDK8/JRE8,
> JDK8/JRE11, JDK11/JRE11 ie. they contained relevant SAN entries in each case.
> *Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
> attempting to send to local input port with RPG*:
> {code:java}
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> natog0.com:9551 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
> doesn't match any of the subject alternative names: [natog1.com] 2019-08-13
> 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster{code}
> But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
[
https://issues.apache.org/jira/browse/NIFI-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-6561:
---
Description:
When testing Java 11 build compatibility, I found an issue with TLS
certificates when using a remote process group looped back to an input port on
the same cluster. The same certificates were used for JDK8/JRE8, JDK8/JRE11,
JDK11/JRE11 ie. they contained relevant SAN entries in each case.
*Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
attempting to send to local input port with RPG*:
{code:java}
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector Could not communicate with
natog0.com:9551 to determine which nodes exist in the remote NiFi cluster, due
to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
doesn't match any of the subject alternative names: [natog1.com]
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
Group's peers due to Unable to communicate with remote NiFi cluster in order to
determine which nodes exist in the remote cluster{code}
But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
was:
When testing Java 11 build compatibility, I found an issue with TLS
certificates when using a remote process group looped back to an input port on
the same cluster. The same certificates were used for JDK8/JRE8, JDK8/JRE11,
JDK11/JRE11 ie. they contained relevant SAN entries in each case.
*Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
attempting to send to local input port with RPG*:
{noformat}
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector Could not communicate with
natog0.com:9551 to determine which nodes exist in the remote NiFi cluster, due
to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
doesn't match any of the subject alternative names: [natog1.com]
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
Group's peers due to Unable to communicate with remote NiFi cluster in order to
determine which nodes exist in the remote cluster{noformat}
But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
> Certificate compatibility broken for JDK8 build running on JRE11
>
>
> Key: NIFI-6561
> URL: https://issues.apache.org/jira/browse/NIFI-6561
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Security
>Affects Versions: 1.10.0
>Reporter: Nathan Gough
>Priority: Major
> Labels: certificate, tls
>
> When testing Java 11 build compatibility, I found an issue with TLS
> certificates when using a remote process group looped back to an input port
> on the same cluster. The same certificates were used for JDK8/JRE8,
> JDK8/JRE11, JDK11/JRE11 ie. they contained relevant SAN entries in each case.
> *Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
> attempting to send to local input port with RPG*:
> {code:java}
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> natog0.com:9551 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
> doesn't match any of the subject alternative names: [natog1.com]
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster{code}
> But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
[
https://issues.apache.org/jira/browse/NIFI-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-6561:
---
External issue URL: (was: https://issues.apache.org/jira/browse/NIFI-5174)
> Certificate compatibility broken for JDK8 build running on JRE11
>
>
> Key: NIFI-6561
> URL: https://issues.apache.org/jira/browse/NIFI-6561
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Security
>Affects Versions: 1.10.0
>Reporter: Nathan Gough
>Priority: Major
> Labels: certificate, tls
>
> When testing Java 11 build compatibility, I found an issue with TLS
> certificates when using a remote process group looped back to an input port
> on the same cluster. The same certificates were used for JDK8/JRE8,
> JDK8/JRE11, JDK11/JRE11 ie. they contained relevant SAN entries in each case.
> *Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
> attempting to send to local input port with RPG*:
> {{}}
> {code:java}
> 2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> natog0.com:9551 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
> doesn't match any of the subject alternative names: [natog1.com] 2019-08-13
> 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster{code}
> But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (NIFI-6561) Certificate compatibility broken for JDK8 build running on JRE11
Nathan Gough created NIFI-6561:
--
Summary: Certificate compatibility broken for JDK8 build running
on JRE11
Key: NIFI-6561
URL: https://issues.apache.org/jira/browse/NIFI-6561
Project: Apache NiFi
Issue Type: Improvement
Components: Security
Affects Versions: 1.10.0
Reporter: Nathan Gough
When testing Java 11 build compatibility, I found an issue with TLS
certificates when using a remote process group looped back to an input port on
the same cluster. The same certificates were used for JDK8/JRE8, JDK8/JRE11,
JDK11/JRE11 ie. they contained relevant SAN entries in each case.
*Building on JDK 1.8.0_172 and run on JRE11.0.5+10 caused exceptions when
attempting to send to local input port with RPG*:
{{}}
{code:java}
2019-08-13 18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector Could not communicate with
natog0.com:9551 to determine which nodes exist in the remote NiFi cluster, due
to javax.net.ssl.SSLPeerUnverifiedException: Certificate for
doesn't match any of the subject alternative names: [natog1.com] 2019-08-13
18:17:07,946 WARN [Http Site-to-Site PeerSelector]
o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@6d5e02f8 Unable to refresh Remote
Group's peers due to Unable to communicate with remote NiFi cluster in order to
determine which nodes exist in the remote cluster{code}
But did not see this error on the matching builds (JDK8/JRE8, JDK11/JRE11).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (NIFI-6559) FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are Missing
[ https://issues.apache.org/jira/browse/NIFI-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909243#comment-16909243 ] Peter Wicks commented on NIFI-6559: --- If I wrote an offline FlowFile Repository compaction utility, and put it in the NiFi utilities, do you think this would be an appropriate option that I could add to it there? > FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are > Missing > - > > Key: NIFI-6559 > URL: https://issues.apache.org/jira/browse/NIFI-6559 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When NiFi is journaling the FlowFile repository changes to disk it sometimes > writes Overflow files if it exceeds a certain memory threshold. > These files are tracked inside of the *.journal files as External File > References. If one of these external file references is deleted or lost the > entire journal fails to recover. > Instead, I feel this should work more like FlowFile's that lose their queue, > or Content in the Content Repository that has lost it's FlowFile. Log it, > and move on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (NIFI-6559) FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are Missing
[ https://issues.apache.org/jira/browse/NIFI-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909243#comment-16909243 ] Peter Wicks edited comment on NIFI-6559 at 8/16/19 5:25 PM: If I wrote an offline FlowFile Repository checkpoint utility, and put it in the NiFi utilities, do you think this would be an appropriate option that I could add to it there? was (Author: patricker): If I wrote an offline FlowFile Repository compaction utility, and put it in the NiFi utilities, do you think this would be an appropriate option that I could add to it there? > FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are > Missing > - > > Key: NIFI-6559 > URL: https://issues.apache.org/jira/browse/NIFI-6559 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When NiFi is journaling the FlowFile repository changes to disk it sometimes > writes Overflow files if it exceeds a certain memory threshold. > These files are tracked inside of the *.journal files as External File > References. If one of these external file references is deleted or lost the > entire journal fails to recover. > Instead, I feel this should work more like FlowFile's that lose their queue, > or Content in the Content Repository that has lost it's FlowFile. Log it, > and move on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (NIFI-6560) messages.es.xlf 404
[ https://issues.apache.org/jira/browse/NIFI-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-6560: -- Status: Patch Available (was: In Progress) > messages.es.xlf 404 > --- > > Key: NIFI-6560 > URL: https://issues.apache.org/jira/browse/NIFI-6560 > Project: Apache NiFi > Issue Type: Bug > Components: FDS >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: fds-0.3 > > Attachments: 62911954-39f1da80-bdc1-11e9-8065-747d3d7b1cd4.png, > Screen Shot 2019-08-16 at 12.59.28 PM.png > > Time Spent: 10m > Remaining Estimate: 0h > > To test, configure your browser language to Spanish: > > !62911954-39f1da80-bdc1-11e9-8065-747d3d7b1cd4.png! > > Build and start the webpack development server. You will see the network > error: > > !Screen Shot 2019-08-16 at 12.59.28 PM.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-fds] scottyaslan opened a new pull request #53: [NIFI-6560] update webapp to useHash routing strategy, request locale…
scottyaslan opened a new pull request #53: [NIFI-6560] update webapp to useHash routing strategy, request locale… URL: https://github.com/apache/nifi-fds/pull/53 … files as text Thank you for submitting a contribution to Apache NiFi Flow Design System. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with either NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you ensured that a full build and that the full suite of unit tests is executed via npm run clean:install at the root nifi-fds folder? - [ ] Have you written or updated the Apache NiFi Flow Design System demo application to demonstrate any new functionality, provide examples of usage, and to verify your changes via npm start at the nifi-fds/target folder? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-fds? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-fds? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6559) FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are Missing
[ https://issues.apache.org/jira/browse/NIFI-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909231#comment-16909231 ] Mark Payne commented on NIFI-6559: -- I don't have any solution off the top of my head. The question I would have is what happened to the file? The way that the logic behaves, in order for the overflow file to be considered part of the repo, the entire file must be entirely written and then fsync'ed to ensure that all contents have been flushed to disk. Only then do we update the FlowFile Repository to point to it because we know the file is in a good state. If arbitrary parts of the store's state/data are manually (or otherwise) deleted outside of the application, I would not expect it to start up and keep functioning, just as I would not expect a database to function properly if a user started deleting some of its files. > FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are > Missing > - > > Key: NIFI-6559 > URL: https://issues.apache.org/jira/browse/NIFI-6559 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When NiFi is journaling the FlowFile repository changes to disk it sometimes > writes Overflow files if it exceeds a certain memory threshold. > These files are tracked inside of the *.journal files as External File > References. If one of these external file references is deleted or lost the > entire journal fails to recover. > Instead, I feel this should work more like FlowFile's that lose their queue, > or Content in the Content Repository that has lost it's FlowFile. Log it, > and move on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (NIFI-6560) messages.es.xlf 404
Scott Aslan created NIFI-6560: - Summary: messages.es.xlf 404 Key: NIFI-6560 URL: https://issues.apache.org/jira/browse/NIFI-6560 Project: Apache NiFi Issue Type: Bug Components: FDS Reporter: Scott Aslan Assignee: Scott Aslan Fix For: fds-0.3 Attachments: 62911954-39f1da80-bdc1-11e9-8065-747d3d7b1cd4.png, Screen Shot 2019-08-16 at 12.59.28 PM.png To test, configure your browser language to Spanish: !62911954-39f1da80-bdc1-11e9-8065-747d3d7b1cd4.png! Build and start the webpack development server. You will see the network error: !Screen Shot 2019-08-16 at 12.59.28 PM.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (NIFIREG-305) Update frontend L & N
[ https://issues.apache.org/jira/browse/NIFIREG-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFIREG-305. - Resolution: Fixed > Update frontend L & N > - > > Key: NIFIREG-305 > URL: https://issues.apache.org/jira/browse/NIFIREG-305 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: 0.5.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Remove the corresponding Apache-2.0 license deps from our LICENSE file and > only if the have a copyright to carry forward do we need to say anything in > our NOTICE file. > MIT/BSD things we want in our LICENSE -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-registry] asfgit closed pull request #217: [NIFIREG-305] update frontend L & N
asfgit closed pull request #217: [NIFIREG-305] update frontend L & N URL: https://github.com/apache/nifi-registry/pull/217 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] mcgilman commented on issue #217: [NIFIREG-305] update frontend L & N
mcgilman commented on issue #217: [NIFIREG-305] update frontend L & N URL: https://github.com/apache/nifi-registry/pull/217#issuecomment-522072738 Thanks @scottyaslan! This has been merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] mcgilman commented on issue #217: [NIFIREG-305] update frontend L & N
mcgilman commented on issue #217: [NIFIREG-305] update frontend L & N URL: https://github.com/apache/nifi-registry/pull/217#issuecomment-522068410 Will review... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f…
bakaid commented on a change in pull request #610: MINIFICPP-814 - Fixed
ListenHTTP and HTTPClient bugs, created tests f…
URL: https://github.com/apache/nifi-minifi-cpp/pull/610#discussion_r314792013
##
File path: extensions/http-curl/client/HTTPClient.cpp
##
@@ -148,6 +161,52 @@ void HTTPClient::setDisableHostVerification() {
curl_easy_setopt(http_session_, CURLOPT_SSL_VERIFYHOST, 0L);
}
+bool HTTPClient::setSpecificSSLVersion(SSLVersion specific_version) {
+#if CURL_AT_LEAST_VERSION(7, 54, 0)
+ CURLcode ret = CURLE_UNKNOWN_OPTION;
+ switch (specific_version) {
+case SSLVersion::SSLv2:
+ ret = curl_easy_setopt(http_session_, CURLOPT_SSLVERSION,
CURL_SSLVERSION_SSLv2);
Review comment:
@alopresto @thenatog Since then we talked about this with @phrocker and
agreed that the full modifications recommended could be part of 0.7.0. I have
already created a follow-up ticket for that:
https://issues.apache.org/jira/browse/MINIFICPP-1005
(Also, the code this conversation is at is outdated, the actual version only
allows >=TLSv1.0 for this function).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f…
bakaid commented on a change in pull request #610: MINIFICPP-814 - Fixed
ListenHTTP and HTTPClient bugs, created tests f…
URL: https://github.com/apache/nifi-minifi-cpp/pull/610#discussion_r314792013
##
File path: extensions/http-curl/client/HTTPClient.cpp
##
@@ -148,6 +161,52 @@ void HTTPClient::setDisableHostVerification() {
curl_easy_setopt(http_session_, CURLOPT_SSL_VERIFYHOST, 0L);
}
+bool HTTPClient::setSpecificSSLVersion(SSLVersion specific_version) {
+#if CURL_AT_LEAST_VERSION(7, 54, 0)
+ CURLcode ret = CURLE_UNKNOWN_OPTION;
+ switch (specific_version) {
+case SSLVersion::SSLv2:
+ ret = curl_easy_setopt(http_session_, CURLOPT_SSLVERSION,
CURL_SSLVERSION_SSLv2);
Review comment:
@alopresto @thenatog Since then we talked about this with @phrocker and
agreed that the full modifications recommended could be part of 0.7.0. I have
already created a follow-up ticket for that:
https://issues.apache.org/jira/browse/MINIFICPP-1005
(Also, the code this conversation is at is outdated, the actual versions
only allows >=TLSv1.0 for this function).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f…
bakaid commented on a change in pull request #610: MINIFICPP-814 - Fixed
ListenHTTP and HTTPClient bugs, created tests f…
URL: https://github.com/apache/nifi-minifi-cpp/pull/610#discussion_r314792013
##
File path: extensions/http-curl/client/HTTPClient.cpp
##
@@ -148,6 +161,52 @@ void HTTPClient::setDisableHostVerification() {
curl_easy_setopt(http_session_, CURLOPT_SSL_VERIFYHOST, 0L);
}
+bool HTTPClient::setSpecificSSLVersion(SSLVersion specific_version) {
+#if CURL_AT_LEAST_VERSION(7, 54, 0)
+ CURLcode ret = CURLE_UNKNOWN_OPTION;
+ switch (specific_version) {
+case SSLVersion::SSLv2:
+ ret = curl_easy_setopt(http_session_, CURLOPT_SSLVERSION,
CURL_SSLVERSION_SSLv2);
Review comment:
@alopresto @thenatog Since then we talked about this with @phrocker and
agreed that the full modifications recommended could be part of 0.7.0. I have
already created a follow-up ticket for that:
https://issues.apache.org/jira/browse/MINIFICPP-1005
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (NIFIREG-307) Update JS dependencies
[ https://issues.apache.org/jira/browse/NIFIREG-307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFIREG-307. - Resolution: Fixed Fix Version/s: 0.5.0 > Update JS dependencies > -- > > Key: NIFIREG-307 > URL: https://issues.apache.org/jira/browse/NIFIREG-307 > Project: NiFi Registry > Issue Type: Bug >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: 0.5.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-registry] mcgilman commented on issue #218: [NIFIREG-307] update JS dependencies
mcgilman commented on issue #218: [NIFIREG-307] update JS dependencies URL: https://github.com/apache/nifi-registry/pull/218#issuecomment-522064986 Thanks @scottyaslan! This has been merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] asfgit closed pull request #218: [NIFIREG-307] update JS dependencies
asfgit closed pull request #218: [NIFIREG-307] update JS dependencies URL: https://github.com/apache/nifi-registry/pull/218 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6559) FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are Missing
[ https://issues.apache.org/jira/browse/NIFI-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909158#comment-16909158 ] Peter Wicks commented on NIFI-6559: --- [~markap14] I don't disagree with any of the downsides/issues you outlined. Do you have an idea for an option to work with this though? Because right now the only option for fixing a corrupt journal is dropping the entire journal with all changes. > FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are > Missing > - > > Key: NIFI-6559 > URL: https://issues.apache.org/jira/browse/NIFI-6559 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When NiFi is journaling the FlowFile repository changes to disk it sometimes > writes Overflow files if it exceeds a certain memory threshold. > These files are tracked inside of the *.journal files as External File > References. If one of these external file references is deleted or lost the > entire journal fails to recover. > Instead, I feel this should work more like FlowFile's that lose their queue, > or Content in the Content Repository that has lost it's FlowFile. Log it, > and move on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi] tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath set value support
tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath set value support URL: https://github.com/apache/nifi/pull/3646#discussion_r314782768 ## File path: nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/JsonPathSetEvaluator.java ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.attribute.expression.language.evaluation.functions; + +import com.jayway.jsonpath.DocumentContext; +import com.jayway.jsonpath.JsonPath; +import org.apache.nifi.attribute.expression.language.EvaluationContext; +import org.apache.nifi.attribute.expression.language.evaluation.Evaluator; +import org.apache.nifi.attribute.expression.language.evaluation.QueryResult; +import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult; + +/** + * JsonPathSetEvaluator allows setting values at the specified path Review comment: I would add "at the specifed _**existing**_ path" or somehow make it clear this can't be used to enrich a JSON This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath set value support
tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath
set value support
URL: https://github.com/apache/nifi/pull/3646#discussion_r314783274
##
File path:
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/JsonPathSetEvaluator.java
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import com.jayway.jsonpath.DocumentContext;
+import com.jayway.jsonpath.JsonPath;
+import org.apache.nifi.attribute.expression.language.EvaluationContext;
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+
+/**
+ * JsonPathSetEvaluator allows setting values at the specified path
+ */
+public class JsonPathSetEvaluator extends JsonPathBaseEvaluator {
+
+protected Evaluator valueEvaluator;
+
+public JsonPathSetEvaluator(final Evaluator subject, final
Evaluator jsonPathExp, final Evaluator valueEvaluator) {
+super(subject, jsonPathExp);
+this.valueEvaluator = valueEvaluator;
+}
+
+@Override
+public QueryResult evaluate(EvaluationContext context) {
+DocumentContext documentContext = getDocumentContext(context);
+final JsonPath compiledJsonPath = getJsonPath(context);;
Review comment:
Minor: double semicolons
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath set value support
tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath
set value support
URL: https://github.com/apache/nifi/pull/3646#discussion_r314784122
##
File path:
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/JsonPathSetEvaluator.java
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import com.jayway.jsonpath.DocumentContext;
+import com.jayway.jsonpath.JsonPath;
+import org.apache.nifi.attribute.expression.language.EvaluationContext;
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+
+/**
+ * JsonPathSetEvaluator allows setting values at the specified path
+ */
+public class JsonPathSetEvaluator extends JsonPathBaseEvaluator {
+
+protected Evaluator valueEvaluator;
+
+public JsonPathSetEvaluator(final Evaluator subject, final
Evaluator jsonPathExp, final Evaluator valueEvaluator) {
+super(subject, jsonPathExp);
+this.valueEvaluator = valueEvaluator;
+}
+
+@Override
+public QueryResult evaluate(EvaluationContext context) {
+DocumentContext documentContext = getDocumentContext(context);
+final JsonPath compiledJsonPath = getJsonPath(context);;
+
+final Object value = valueEvaluator.evaluate(context).getValue();
+
+String result = null;
Review comment:
Minor: No need to initialise as null.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath set value support
tpalfy commented on a change in pull request #3646: NIFI-6546 - Add JsonPath
set value support
URL: https://github.com/apache/nifi/pull/3646#discussion_r314767395
##
File path:
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/compile/ExpressionCompiler.java
##
@@ -919,6 +924,29 @@ private DateEvaluator toDateEvaluator(final Evaluator
evaluator, final String
return addToken(new
JsonPathDeleteEvaluator(toStringEvaluator(subjectEvaluator),
toStringEvaluator(argEvaluators.get(0), "first
argument to jsonPathDelete")), "jsonPathDelete");
}
+case JSON_PATH_SET: {
+verifyArgCount(argEvaluators, 2, "jsonPathSet");
+Evaluator valueEvaluator = null;
+Evaluator argValueEvaluator = argEvaluators.get(1);
+String location = "second argument to jsonPathSet";
+if (argValueEvaluator instanceof StringEvaluator) {
+valueEvaluator = toStringEvaluator(argValueEvaluator,
location);
+} else if (argValueEvaluator instanceof DecimalEvaluator) {
+valueEvaluator = toDecimalEvaluator(argValueEvaluator,
location);
+} else if (argValueEvaluator instanceof NumberEvaluator) {
+valueEvaluator = toNumberEvaluator(argValueEvaluator,
location);
+} else if (argValueEvaluator instanceof WholeNumberEvaluator) {
+valueEvaluator = toWholeNumberEvaluator(argValueEvaluator,
location);
+} else if (argValueEvaluator instanceof BooleanEvaluator) {
+valueEvaluator = toBooleanEvaluator(argValueEvaluator,
location);
+} else {
+throw new
AttributeExpressionLanguageParsingException("Cannot implicitly convert Data
Type " + argValueEvaluator.getResultType() + " to " + ResultType.WHOLE_NUMBER
Review comment:
The `" to " + ResultType.WHOLE_NUMBER` part was probably an unintentional
remainder of a copy.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] thenatog commented on a change in pull request #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f…
thenatog commented on a change in pull request #610: MINIFICPP-814 - Fixed
ListenHTTP and HTTPClient bugs, created tests f…
URL: https://github.com/apache/nifi-minifi-cpp/pull/610#discussion_r314781748
##
File path: extensions/http-curl/client/HTTPClient.cpp
##
@@ -148,6 +161,52 @@ void HTTPClient::setDisableHostVerification() {
curl_easy_setopt(http_session_, CURLOPT_SSL_VERIFYHOST, 0L);
}
+bool HTTPClient::setSpecificSSLVersion(SSLVersion specific_version) {
+#if CURL_AT_LEAST_VERSION(7, 54, 0)
+ CURLcode ret = CURLE_UNKNOWN_OPTION;
+ switch (specific_version) {
+case SSLVersion::SSLv2:
+ ret = curl_easy_setopt(http_session_, CURLOPT_SSLVERSION,
CURL_SSLVERSION_SSLv2);
Review comment:
@alopresto, what are your opinions?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (NIFI-6559) FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are Missing
[ https://issues.apache.org/jira/browse/NIFI-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909133#comment-16909133 ] Mark Payne commented on NIFI-6559: -- [~patricker] I don't think this is a change that we want to make. If a FlowFile's queue is missing, then the FlowFile is dropped from the system. However, if an Overflow file is missing, it would not drop the FlowFile from the system - it would simply lose some updates to the FlowFile. Now if this were the last update to the FlowFile, it may not be a big deal. However, if it's not the last update to the FlowFile, this can be a very big deal. For example, consider the following set of updates to the FlowFile Repository, all for FlowFile A: * FlowFile A created. Placed in Queue 123. * Information is extracted from FlowFile and added as attributes. Placed in Queue 456. * Processor X removes any PII information from FlowFile Attributes. Placed on Queue 789. — This is in an overflow file that gets dropped. * FlowFile routing done and placed on Queue 222. Now, consider that NiFi is restarted. Upon restart, we have lost the update to the attributes that removed PII. But we've placed the FlowFile, with PII, on Queue 222. Upon restart, we've now loaded up this FlowFile into a queue that it should never have been able to get to without first going through some other process. Essentially, we have caused the FlowFile to skip some processor in the flow. This could result in data loss, or it can result in data leakage, corruption, or several other possible problems. While it would be acceptable to lose the last update to a FlowFile, which would result in essentially replaying the FlowFile from some point in the flow, it is imperative that the FlowFile repository never allow an intermediate update to a FlowFile to be lost. > FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are > Missing > - > > Key: NIFI-6559 > URL: https://issues.apache.org/jira/browse/NIFI-6559 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When NiFi is journaling the FlowFile repository changes to disk it sometimes > writes Overflow files if it exceeds a certain memory threshold. > These files are tracked inside of the *.journal files as External File > References. If one of these external file references is deleted or lost the > entire journal fails to recover. > Instead, I feel this should work more like FlowFile's that lose their queue, > or Content in the Content Repository that has lost it's FlowFile. Log it, > and move on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-minifi-cpp] phrocker commented on a change in pull request #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f…
phrocker commented on a change in pull request #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f… URL: https://github.com/apache/nifi-minifi-cpp/pull/610#discussion_r304559838 ## File path: extensions/http-curl/CMakeLists.txt ## @@ -24,11 +24,6 @@ include_directories(protocols client processors sitetosite) file(GLOB SOURCES "*.cpp" "protocols/*.cpp" "client/*.cpp" "processors/*.cpp" "sitetosite/*.cpp") -if (USE_CURL_NSS) Review comment: Di This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] phrocker commented on issue #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f…
phrocker commented on issue #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f… URL: https://github.com/apache/nifi-minifi-cpp/pull/610#issuecomment-522044392 @bakaid will take another look and merge after some other test prs are merged. thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (NIFIREG-295) Add support for proxying via Apache Knox
[ https://issues.apache.org/jira/browse/NIFIREG-295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFIREG-295. - Resolution: Fixed Fix Version/s: 0.5.0 > Add support for proxying via Apache Knox > > > Key: NIFIREG-295 > URL: https://issues.apache.org/jira/browse/NIFIREG-295 > Project: NiFi Registry > Issue Type: New Feature >Reporter: Kevin Risden >Assignee: Koji Kawamura >Priority: Major > Fix For: 0.5.0 > > Time Spent: 6h 20m > Remaining Estimate: 0h > > Apache NiFi supports proxying via Apache Knox, but NiFi Registry does not. > This would make single sign on between the different servers seamless. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-registry] asfgit closed pull request #211: NIFIREG-295 Add support for proxying via Apache Knox
asfgit closed pull request #211: NIFIREG-295 Add support for proxying via Apache Knox URL: https://github.com/apache/nifi-registry/pull/211 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (MINIFICPP-1005) Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections
[ https://issues.apache.org/jira/browse/MINIFICPP-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909104#comment-16909104 ] Mr TheSegfault commented on MINIFICPP-1005: --- Good call. I don't know if backporting is necessary. We can make it very clear in our documentation that this is the case for 0.7.0 moving forward. That also sets a precedent that we may need to go back further; however, having a separate ticket, but we typically squash when merging – we'll have to make sure not to do that. > Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections > --- > > Key: MINIFICPP-1005 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1005 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Daniel Bakai >Assignee: Daniel Bakai >Priority: Blocker > Fix For: 0.7.0 > > > This affects multiple processors, like ListenHTTP and InvokeHTTP, and C2. All > instances should be considered. > I am creating this as a separate blocker ticket and not including it in > MINIFICPP-814 to make that eaiser to backport to 0.6.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-registry] bbende commented on issue #211: NIFIREG-295 Add support for proxying via Apache Knox
bbende commented on issue #211: NIFIREG-295 Add support for proxying via Apache Knox URL: https://github.com/apache/nifi-registry/pull/211#issuecomment-522038489 Looks like there is consensus that this is good to go, thanks @ijokarumawak !! Will merge shortly This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (NIFI-6475) NoClassDefFoundError when running Atlas integration from Docker container
[
https://issues.apache.org/jira/browse/NIFI-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Flowers resolved NIFI-6475.
--
Resolution: Invalid
> NoClassDefFoundError when running Atlas integration from Docker container
> -
>
> Key: NIFI-6475
> URL: https://issues.apache.org/jira/browse/NIFI-6475
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.8.0
>Reporter: Justin Flowers
>Priority: Major
>
> Running the Atlas integration in a Nifi Docker container throws this
> exception after configuring the ReportLineageToAtlas Reporting Task and
> staring it:
> {code:java}
> 2019-07-23 13:00:05,753 ERROR [Timer-Driven Process Thread-3]
> o.a.n.a.reporting.ReportLineageToAtlas
> ReportLineageToAtlas[id=1f01a738-016c-1000--ada9f376] Failed to
> invoke @OnScheduled method due to java.lang.NoClassDefFoundError: Could not
> initialize class org.apache.nifi.atlas.hook.NiFiAtlasHook: {}
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.nifi.atlas.hook.NiFiAtlasHook
> at
> org.apache.nifi.atlas.reporting.ReportLineageToAtlas.initLineageStrategy(ReportLineageToAtlas.java:453)
> at
> org.apache.nifi.atlas.reporting.ReportLineageToAtlas.setup(ReportLineageToAtlas.java:448)
> at sun.reflect.GeneratedMethodAccessor366.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75)
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52)
> at
> org.apache.nifi.controller.scheduling.StandardProcessScheduler$2.run(StandardProcessScheduler.java:214)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
> I tried running on empty standalone Docker deploys and on empty standalone VM
> filesystem deploys, but I only see this issue running from a container.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [nifi-minifi-cpp] bakaid commented on issue #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f…
bakaid commented on issue #610: MINIFICPP-814 - Fixed ListenHTTP and HTTPClient bugs, created tests f… URL: https://github.com/apache/nifi-minifi-cpp/pull/610#issuecomment-522035977 I have added https://issues.apache.org/jira/browse/MINIFICPP-1005 as a blocker follow-up task about TLS versions. I think it's better not to include that in this PR, so that it will be easier to backport to 0.6.0 for 0.6.1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (MINIFICPP-1005) Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections
Daniel Bakai created MINIFICPP-1005: --- Summary: Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections Key: MINIFICPP-1005 URL: https://issues.apache.org/jira/browse/MINIFICPP-1005 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Daniel Bakai Assignee: Daniel Bakai Fix For: 0.7.0 This affects multiple processors, like ListenHTTP and InvokeHTTP, and C2. All instances should be considered. I am creating this as a separate blocker ticket and not including it in MINIFICPP-814 to make that eaiser to backport to 0.6.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi] patricker opened a new pull request #3655: NIFI-6559 FlowFile Repo Journal Recovery Should not Fail if External …
patricker opened a new pull request #3655: NIFI-6559 FlowFile Repo Journal
Recovery Should not Fail if External …
URL: https://github.com/apache/nifi/pull/3655
…Overflow Files are Missing
Description of PR
When NiFi is journaling the FlowFile repository changes to disk it sometimes
writes Overflow files if it exceeds a certain memory threshold.
These files are tracked inside of the *.journal files as External File
References. If one of these external file references is deleted or lost the
entire journal fails to recover.
Instead, I feel this should work more like FlowFile's that lose their queue,
or Content in the Content Repository that has lost it's FlowFile. Log it, and
move on.
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [x] Have you written or updated unit tests to verify your changes?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (NIFI-6559) FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are Missing
Peter Wicks created NIFI-6559: - Summary: FlowFile Repo Journal Recovery Should not Fail if External Overflow Files are Missing Key: NIFI-6559 URL: https://issues.apache.org/jira/browse/NIFI-6559 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Peter Wicks Assignee: Peter Wicks When NiFi is journaling the FlowFile repository changes to disk it sometimes writes Overflow files if it exceeds a certain memory threshold. These files are tracked inside of the *.journal files as External File References. If one of these external file references is deleted or lost the entire journal fails to recover. Instead, I feel this should work more like FlowFile's that lose their queue, or Content in the Content Repository that has lost it's FlowFile. Log it, and move on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-minifi-cpp] nghiaxlee commented on issue #621: MINIFICPP-959: Review librdkafka thread safety
nghiaxlee commented on issue #621: MINIFICPP-959: Review librdkafka thread safety URL: https://github.com/apache/nifi-minifi-cpp/pull/621#issuecomment-522029091 Yes, I did check it with a docker broker This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] mcgilman commented on issue #218: [NIFIREG-307] update JS dependencies
mcgilman commented on issue #218: [NIFIREG-307] update JS dependencies URL: https://github.com/apache/nifi-registry/pull/218#issuecomment-522023340 Will review... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite
server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314729035
##
File path:
nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java
##
@@ -1224,6 +1249,24 @@ public void testReceiveUnknownPort() throws Exception {
}
}
+@Test
+public void testReceiveEmptyPort() throws Exception {
+
+try (
+SiteToSiteClient client = getDefaultBuilder()
+.portName("empty-output-running")
+.build()
+) {
+try {
+client.createTransaction(TransferDirection.RECEIVE);
+fail();
+} catch (NoContentException e) {
+logger.info("Exception message: {}", e.getMessage());
+assertTrue(e.getMessage().contains("Server has no flowfiles to
provide"));
Review comment:
The real error message is `"No peers have flowfiles to provide"`.
`assertTrue` checks are not too descriptive when there is a failure, a more
user-friendly assert might be better, something like this:
`assertThat(e.getMessage(), (org.hamcrest.CoreMatchers.)containsString("No
peers have flowfiles to provide"))`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite
server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314721193
##
File path:
nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/http/HttpClient.java
##
@@ -168,6 +173,10 @@ public Transaction createTransaction(final
TransferDirection direction) throws H
try {
transactionUrl = apiClient.initiateTransaction(direction,
portId);
commSession.setUserDn(apiClient.getTrustedPeerDn());
+} catch (final NoContentException e) {
Review comment:
Shouldn't we close `apiClient` here as well?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite
server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314727977
##
File path:
nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java
##
@@ -1224,6 +1249,24 @@ public void testReceiveUnknownPort() throws Exception {
}
}
+@Test
+public void testReceiveEmptyPort() throws Exception {
Review comment:
Not entirely sure how this works but I ran all tests in this class and 3
failed (out of 33).
The CI jobs are inconsistent, I see some run only 5 but one fails with the
same 3 as mine does.
The culprit seems to be this new test and the main cause is that it runs for
almost half a minute and after a while it fails due to a
`java.net.ConnectException` which forces the `HttpClient` to return with a null
transaction. (The other 2 tests succeed on their own so I guess this is a
time-related issue.)
This problem doesn't occur if the the peer is penalised. Probably has
something to do with the fact the `PeerSelector.getNextPeerStatus` in the
`HttpClient` just keeps returning peers if they are not penalised.
Maybe both
```java
apiClient.close();
peerSelector.penalize(peer, penaltyMillis);
```
would be in order when `NoContentException` comes in `HttpClient`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite
server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314729035
##
File path:
nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java
##
@@ -1224,6 +1249,24 @@ public void testReceiveUnknownPort() throws Exception {
}
}
+@Test
+public void testReceiveEmptyPort() throws Exception {
+
+try (
+SiteToSiteClient client = getDefaultBuilder()
+.portName("empty-output-running")
+.build()
+) {
+try {
+client.createTransaction(TransferDirection.RECEIVE);
+fail();
+} catch (NoContentException e) {
+logger.info("Exception message: {}", e.getMessage());
+assertTrue(e.getMessage().contains("Server has no flowfiles to
provide"));
Review comment:
The real error message is `"No peers have flowfiles to provide"`.
`assertTrue` checks are not too descriptive when there is a failure, a more
user-friendly assert might be better, something like this:
`assertThat(e.getMessage(),
(org.hamcrest.CoreMatchers.)containsString("Server has no flowfiles to
provide"))`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is ava…
tpalfy commented on a change in pull request #3647: NIFI-6530 - HTTP SiteToSite
server returns 201 in case no data is ava…
URL: https://github.com/apache/nifi/pull/3647#discussion_r314736158
##
File path:
nifi-stateless/nifi-stateless-core/src/main/java/org/apache/nifi/stateless/core/StatelessRemoteOutputPort.java
##
@@ -139,7 +140,11 @@ public boolean runRecursive(final Queue
queue) {
transaction.confirm();
transaction.complete();
-} catch (final Exception e) {
+} catch (final NoContentException e) {
+getLogger().info("No flowfiles to receive");
Review comment:
Not sure if `info` logging is intended. In other places the "No flowfile"
case only warranted a debug level log.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] phrocker commented on issue #595: Add windows tests
phrocker commented on issue #595: Add windows tests URL: https://github.com/apache/nifi-minifi-cpp/pull/595#issuecomment-522021954 @apiri thanks. pushed another fix for a unit test and to remove cout statements. will wait for a +1 to merge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFIREG-308) UI - update webpack dev server build to run secure
Scott Aslan created NIFIREG-308: --- Summary: UI - update webpack dev server build to run secure Key: NIFIREG-308 URL: https://issues.apache.org/jira/browse/NIFIREG-308 Project: NiFi Registry Issue Type: Bug Reporter: Scott Aslan The webpack development build does not support proxy'ing to a secured nifi registry. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #629: MINIFICPP-1002 - Add SourceInitiatedSubscriptionListener processor
bakaid commented on a change in pull request #629: MINIFICPP-1002 - Add
SourceInitiatedSubscriptionListener processor
URL: https://github.com/apache/nifi-minifi-cpp/pull/629#discussion_r314719220
##
File path: centos.sh
##
@@ -155,6 +155,8 @@ build_deps(){
INSTALLED+=("bzip2-devel")
elif [ "$FOUND_VALUE" = "libssh2" ]; then
INSTALLED+=("libssh2-devel")
+elif [ "$FOUND_VALUE" = "libxml2" ]; then
Review comment:
We can also bundle libxml2, but it is a pretty common library, so I left it
an external dependency for the time being.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #629: MINIFICPP-1002 - Add SourceInitiatedSubscriptionListener processor
bakaid commented on a change in pull request #629: MINIFICPP-1002 - Add
SourceInitiatedSubscriptionListener processor
URL: https://github.com/apache/nifi-minifi-cpp/pull/629#discussion_r314718886
##
File path: thirdparty/openwsman/openwsman.patch
##
@@ -0,0 +1,102 @@
+diff -rupN openwsman-2.6.9/CMakeLists.txt
openwsman-2.6.9-patched/CMakeLists.txt
+--- openwsman-2.6.9/CMakeLists.txt 2018-11-20 09:09:43.0 +0100
openwsman-2.6.9-patched/CMakeLists.txt 2019-08-01 15:48:44.0
+0200
+@@ -22,7 +22,7 @@ if(COMMAND cmake_policy)
+ cmake_policy(SET CMP0046 OLD)
+ endif ( POLICY CMP0046 )
+ endif(COMMAND cmake_policy)
+-
++
+ # where to look first for cmake modules, before ${CMAKE_ROOT}/Modules/ is
checked
+ SET(CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake/modules ${CMAKE_MODULE_PATH})
+
+@@ -84,7 +84,7 @@ MACRO(NO_HEADERS_WARNING_EXPL optionName
+MESSAGE(STATUS "")
+SET(HAVE_WARNINGS YES)
+ ENDMACRO(NO_HEADERS_WARNING_EXPL)
+-
++
+ # Macro for error message when header files not present, but build option
+ # not explicitly declared
+ MACRO(NO_HEADERS_WARNING optionName pkgName)
+@@ -170,6 +170,12 @@ INCLUDE(FindOpenSSL)
+ IF(OPENSSL_FOUND)
+ SET(HAVE_SSL 1)
+ SET(USE_OPENSSL 1)
++ INCLUDE_DIRECTORIES(${OPENSSL_INCLUDE_DIR})
++ message(OpenSSL found)
++ message(OPENSSL_INCLUDE_DIR: ${OPENSSL_INCLUDE_DIR})
++ message(OPENSSL_LIBRARIES: ${OPENSSL_LIBRARIES})
++ELSE(OPENSSL_FOUND)
++ SET(HAVE_SSL 0)
+ ENDIF(OPENSSL_FOUND)
+
+ IF( BUILD_RUBY )
+@@ -260,7 +266,7 @@ ELSE ( NOT CURL_FOUND)
+ STRING(COMPARE LESS ${CURL_VERSION_STRING} "7.12.0" result)
+ IF(result LESS 0)
+ MESSAGE( FATAL_ERROR " curl version ${CURL_VERSION_STRING} is too low,
need 7.12.0 or greater" )
+-ENDIF(result LESS 0)
++ENDIF(result LESS 0)
+ ENDIF(CURL_VERSION_STRING)
+ ENDIF( NOT CURL_FOUND)
+
+@@ -439,7 +445,7 @@ IF(HAVE_SA_LEN)
+ ELSE(HAVE_SA_LEN)
+ SET(HAVE_SA_LEN 0)
+ ENDIF(HAVE_SA_LEN)
+-
++
+ IF (ENABLE_IPV6)
+ # Check if struct sockaddr_in6 contains sin6
+ CHECK_STRUCT_HAS_MEMBER("struct sockaddr_in6" sin6_addr netinet/in.h
HAVE_IPV6)
+@@ -592,7 +598,7 @@ IF(HAVE_WARNINGS)
+ MESSAGE(STATUS " NOTE: ")
+ MESSAGE(STATUS " Warnings occurred during cmake configuration... Please
see output")
+ MESSAGE(STATUS " ***")
+-ENDIF(HAVE_WARNINGS)
++ENDIF(HAVE_WARNINGS)
+
+
+ # RPM SPEC #
+diff -rupN openwsman-2.6.9/src/lib/wsman-soap.c
openwsman-2.6.9-patched/src/lib/wsman-soap.c
+--- openwsman-2.6.9/src/lib/wsman-soap.c 2018-11-20 09:09:43.0
+0100
openwsman-2.6.9-patched/src/lib/wsman-soap.c 2019-08-01
15:28:56.0 +0200
+@@ -935,16 +935,18 @@ unsigned long get_total_enum_context(WsC
+ * to libwsman.la. So when a call is made to the following methods
+ * from the openwsmand binary, they should be present.
+ *
+- * However, if they are dlopened from somewhere other than
++ * However, if they are dlopened from somewhere other than
+ * openwsmand library or linked to some other
+ * binary or shared object, then these methods may or may not be
+ * preset, hence marking them as weak symbols and testing to see
+ * if they are resolved before using them.
+ */
++#if 0
Review comment:
Unfortunately exporting all symbols with whole-archive methods don't work
well with the weak symbol approach this library tries to take, so I had to
disable these symbols entirely.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-6416) WriteAheadFlowFileRepository.swapFlowFilesIn() not removing swapLocationSuffixes
[
https://issues.apache.org/jira/browse/NIFI-6416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon DeVries updated NIFI-6416:
--
Assignee: Brandon DeVries
Status: Patch Available (was: Open)
> WriteAheadFlowFileRepository.swapFlowFilesIn() not removing
> swapLocationSuffixes
>
>
> Key: NIFI-6416
> URL: https://issues.apache.org/jira/browse/NIFI-6416
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.9.2
>Reporter: Brandon DeVries
>Assignee: Brandon DeVries
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> WriteAheadFlowFileRepository maintains a list of swapLocationSuffixes. This
> is added to when records are swapped out, and removed from when records are
> swapped in when the repo is updated[1]. However, the swapFlowFilesIn
> method[2] appears to be adding instead of removing the specified swap
> location.
> Specifically:
> {code:java}
> synchronized (this.swapLocationSuffixes) {
> this.swapLocationSuffixes.add(normalizeSwapLocation(swapLocation));
> }{code}
> Should be:
> {code:java}
> synchronized (this.swapLocationSuffixes) {
> this.swapLocationSuffixes.remove(normalizeSwapLocation(swapLocation));
> }{code}
>
>
> [1]
> [https://github.com/apache/nifi/blob/f08c2ee43f39584b7d4e76d61d7d4aaa889068fd/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/WriteAheadFlowFileRepository.java#L372-L377]
> [2]
> [https://github.com/apache/nifi/blob/f08c2ee43f39584b7d4e76d61d7d4aaa889068fd/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/WriteAheadFlowFileRepository.java#L518]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [nifi] devriesb opened a new pull request #3654: NIFI-6416: Removing swap locations on SWAP_IN
devriesb opened a new pull request #3654: NIFI-6416: Removing swap locations on
SWAP_IN
URL: https://github.com/apache/nifi/pull/3654
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on issue #629: MINIFICPP-1002 - Add SourceInitiatedSubscriptionListener processor
bakaid commented on issue #629: MINIFICPP-1002 - Add SourceInitiatedSubscriptionListener processor URL: https://github.com/apache/nifi-minifi-cpp/pull/629#issuecomment-522006890 This processor has only been tested with a Windows 10 machine, and even there some weird edge cases happen where after reconnection it takes a really long time to resume proper operations, so more testing (and testing at scale) is definitely required. Currently the processor emits flow files as Event XMLs come. I am not sure what the benefits and disadvantages of adding something like a moodycamel::ConcurrentQueue to collect these XMLs and emit them in onTrigger. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid opened a new pull request #629: MINIFICPP-1002 - Add SourceInitiatedSubscriptionListener processor
bakaid opened a new pull request #629: MINIFICPP-1002 - Add SourceInitiatedSubscriptionListener processor URL: https://github.com/apache/nifi-minifi-cpp/pull/629 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-5091) nifi CaptureChangeMySQL should consider timezone
[ https://issues.apache.org/jira/browse/NIFI-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909038#comment-16909038 ] Nicolas De Amicis commented on NIFI-5091: - This bug is on the library mysql-binlog-connector-java on datetime deserialization. I made a pull request on that project: https://github.com/shyiko/mysql-binlog-connector-java/pull/292 > nifi CaptureChangeMySQL should consider timezone > > > Key: NIFI-5091 > URL: https://issues.apache.org/jira/browse/NIFI-5091 > Project: Apache NiFi > Issue Type: Bug >Reporter: Jasee Tao >Priority: Minor > > MySQL's binlog doesn't log timezone。For example if I have a mysql using > timezone CST(GMT+8)，and insert a create_time '2018-04-18 08:00:00’ (already > in CST)，in the binlog the create_time is '2018-04-18 08:00:00' (without > timezone information)，and in the output of CaptureChangeMySQL the create_time > becomes '2018-04-18 16:00:00 CST'。Perhaps should add a configurable > "timezone" property to identify the binlog's timezone? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi-registry] scottyaslan commented on issue #211: NIFIREG-295 Add support for proxying via Apache Knox
scottyaslan commented on issue #211: NIFIREG-295 Add support for proxying via Apache Knox URL: https://github.com/apache/nifi-registry/pull/211#issuecomment-521999355 I have also reviewed this PR and the changes look good. I built it and ran a secured (with Certs) local build and verified the deep links and api calls still work. +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] RynoRn commented on issue #211: NIFIREG-295 Add support for proxying via Apache Knox
RynoRn commented on issue #211: NIFIREG-295 Add support for proxying via Apache Knox URL: https://github.com/apache/nifi-registry/pull/211#issuecomment-521993645 Thanks for helping in @rfellows! The suggested approach looks great. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (MINIFICPP-984) LogAttribute should return to prior behavior or detect text and print it out
[ https://issues.apache.org/jira/browse/MINIFICPP-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Bakai resolved MINIFICPP-984. Resolution: Fixed Fix Version/s: 0.7.0 > LogAttribute should return to prior behavior or detect text and print it out > > > Key: MINIFICPP-984 > URL: https://issues.apache.org/jira/browse/MINIFICPP-984 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Daniel Bakai >Priority: Blocker > Fix For: 0.7.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Noticed by user: change in behavior on log attribute payload log. > Prior behavior on text files printed text. In some cases this may have > issues. I think we took a step forward; however, we should provide hex > printing along with ascii ( leaving what was default previously as the > default ) to be backwards compatible and provide the benefits of hex encoding > ( maybe make it EL based, so it can be per ff ) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [nifi] tpalfy commented on a change in pull request #3652: NIFI-5452: Allow For Ignoring Block Locality When Writing to HDFS
tpalfy commented on a change in pull request #3652: NIFI-5452: Allow For
Ignoring Block Locality When Writing to HDFS
URL: https://github.com/apache/nifi/pull/3652#discussion_r314691315
##
File path:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java
##
@@ -313,8 +328,18 @@ public void process(InputStream in) throws IOException {
if
(conflictResponse.equals(APPEND_RESOLUTION_AV.getValue()) && destinationExists)
{
fos = hdfs.append(copyFile, bufferSize);
} else {
-fos = hdfs.create(tempCopyFile, true,
bufferSize, replication, blockSize);
-}
+ final EnumSet cflags =
EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE);
+
+ final Boolean ignoreLocality =
context.getProperty(IGNORE_LOCALITY).asBoolean();
+ if (ignoreLocality.booleanValue()) {
+
cflags.add(CreateFlag.IGNORE_CLIENT_LOCALITY);
+ }
+
+ fos = hdfs.create(tempCopyFile,
FsCreateModes.applyUMask(FsPermission.getFileDefault(),
+ FsPermission.getUMask(hdfs.getConf())),
cflags, bufferSize, replication, blockSize,
+ null, null);
+ }
Review comment:
Minor: The indentation of this closing curly bracket is off.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
