[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[HandOfGod94]

Github user HandOfGod94 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91012450
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724368#comment-15724368
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user HandOfGod94 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91012450
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724361#comment-15724361
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user HandOfGod94 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91012282
  
--- Diff: README.md ---
@@ -12,7 +12,7 @@
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
-# Apache NiFi [![Build 
Status](https://travis-ci.org/apache/nifi.svg?branch=master)](https://travis-ci.org/apache/nifi)
+# Apache NiFi [![Build 
Status](https://travis-ci.org/HandOfGod94/nifi.svg?branch=NIFI-2961)](https://travis-ci.org/HandOfGod94/nifi)
--- End diff --

Oop's my bad..sorry


> Create EncryptAttribute processor
> -
>
> Key: NIFI-2961
> URL: https://issues.apache.org/jira/browse/NIFI-2961
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Andy LoPresto
>  Labels: attributes, encryption, security
>
> Similar to {{EncryptContent}}, the {{EncryptAttribute}} processor would allow 
> individual (and multiple) flowfile attributes to be encrypted (either 
> in-place or to a new attribute key) with various encryption algorithms (AES, 
> RSA, PBE, and PGP). 
> Specific compatibility with the {{OpenSSL EVP_BytesToKey}}, {{PBKDF2}}, 
> {{scrypt}}, and {{bcrypt}} key derivation functions should be included. 
> The processor should provide the boolean option to encrypt or decrypt (only 
> one operation per instance of the processor). The processor should also allow 
> Base64 encoding (aka ASCII armor) for the encrypted attributes to prevent 
> byte escaping/data loss. 
> If [dangerous processor 
> annotations|https://cwiki.apache.org/confluence/display/NIFI/Security+Feature+Roadmap]
>  are introduced, this processor should be marked as such and the 
> corresponding attribute protection (i.e. provenance before/after, etc.) 
> should be applied. 
> Originally requested in this [Stack Overflow 
> question|https://stackoverflow.com/questions/40294945/nifi-encrypt-json].  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[HandOfGod94]

Github user HandOfGod94 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91012282
  
--- Diff: README.md ---
@@ -12,7 +12,7 @@
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
-# Apache NiFi [![Build 
Status](https://travis-ci.org/apache/nifi.svg?branch=master)](https://travis-ci.org/apache/nifi)
+# Apache NiFi [![Build 
Status](https://travis-ci.org/HandOfGod94/nifi.svg?branch=NIFI-2961)](https://travis-ci.org/HandOfGod94/nifi)
--- End diff --

Oop's my bad..sorry


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2656) Allow bootstrap process to prompt for password/key

[Andy LoPresto (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724336#comment-15724336
 ] 

Andy LoPresto commented on NIFI-2656:
-

Thanks Anders. I will take a look at this and provide feedback. In the future, 
you are also welcome to submit a GitHub Pull Request. Submitting a patch here 
is not wrong by any means, but the PRs usually get more visibility and provide 
some reviewing features that some committers rely on. 

> Allow bootstrap process to prompt for password/key
> --
>
> Key: NIFI-2656
> URL: https://issues.apache.org/jira/browse/NIFI-2656
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Configuration, Core Framework
>Affects Versions: 1.0.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Minor
>  Labels: bootstrap, config, encryption, security
> Fix For: 1.2.0
>
> Attachments: NIFI-2656.-K_support.1.patch
>
>
> The bootstrap process {{RunNiFi.java}} is currently responsible for reading 
> the key from {{bootstrap.conf}} and sending it to the running NiFi process 
> {{NiFi.java}} to be used for sensitive property decryption. This exposes the 
> key in two places:
> * Plaintext in {{bootstrap.conf}}
> * In the process invocation
> Running the following command ({{ps -aef | grep -i nifi}}) will result in the 
> following output:
> {code}
> ...
>   501 11597 11596   0  6:51PM ttys0010:08.55 
> /Users/alopresto/.jenv/versions/1.8/bin/java -classpath 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./conf:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/bcprov-jdk15on-1.54.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/commons-lang3-3.4.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/jcl-over-slf4j-1.7.12.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/jul-to-slf4j-1.7.12.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/log4j-over-slf4j-1.7.12.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/logback-classic-1.1.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/logback-core-1.1.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/nifi-api-1.0.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/nifi-documentation-1.0.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/nifi-framework-api-1.0.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/nifi-nar-utils-1.0.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/nifi-properties-1.0.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/nifi-properties-loader-1.0.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/nifi-runtime-1.0.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./lib/slf4j-api-1.7.12.jar
>  -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m 
> -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
> -Djava.awt.headless=true -XX:+UseG1GC 
> -Djava.protocol.handler.pkgs=sun.net.www.protocol 
> -Dnifi.properties.file.path=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/./conf/nifi.properties
>  -Dnifi.bootstrap.listen.port=58213 -Dapp=NiFi 
> -Dorg.apache.nifi.bootstrap.config.log.dir=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.0.0-SNAPSHOT-bin/nifi-1.0.0-SNAPSHOT/logs
>  org.apache.nifi.NiFi -k 
> 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210
> ...
> {code}
> To allow for a more secure invocation, the NiFi process could pause and 
> prompt for the password/key entry in a secure console if it is not provided 
> in the invocation arguments from bootstrap (or if a special flag is 
> provided). While this would require manual intervention to start the process, 
> it would not be default behavior. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724319#comment-15724319
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91003876
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724326#comment-15724326
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91009968
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91009550
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724331#comment-15724331
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010400
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724323#comment-15724323
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91009609
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/resources/docs/org/apache/nifi/processors/standard/EncryptAttributes/additionalDetails.html
 ---
@@ -0,0 +1,30 @@
+
--- End diff --

The message contained here is not relevant to `EncryptAttributes` so this 
file can be removed. 


> Create EncryptAttribute processor
> -
>
> Key: NIFI-2961
> URL: https://issues.apache.org/jira/browse/NIFI-2961
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Andy LoPresto
>  Labels: attributes, encryption, security
>
> Similar to {{EncryptContent}}, the {{EncryptAttribute}} processor would allow 
> individual (and multiple) flowfile attributes to be encrypted (either 
> in-place or to a new attribute key) with various encryption algorithms (AES, 
> RSA, PBE, and PGP). 
> Specific compatibility with the {{OpenSSL EVP_BytesToKey}}, {{PBKDF2}}, 
> {{scrypt}}, and {{bcrypt}} key derivation functions should be included. 
> The processor should provide the boolean option to encrypt or decrypt (only 
> one operation per instance of the processor). The processor should also allow 
> Base64 encoding (aka ASCII armor) for the encrypted attributes to prevent 
> byte escaping/data loss. 
> If [dangerous processor 
> annotations|https://cwiki.apache.org/confluence/display/NIFI/Security+Feature+Roadmap]
>  are introduced, this processor should be marked as such and the 
> corresponding attribute protection (i.e. provenance before/after, etc.) 
> should be applied. 
> Originally requested in this [Stack Overflow 
> question|https://stackoverflow.com/questions/40294945/nifi-encrypt-json].  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724328#comment-15724328
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90783308
  
--- Diff: README.md ---
@@ -12,7 +12,7 @@
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
-# Apache NiFi [![Build 
Status](https://travis-ci.org/apache/nifi.svg?branch=master)](https://travis-ci.org/apache/nifi)
+# Apache NiFi [![Build 
Status](https://travis-ci.org/HandOfGod94/nifi.svg?branch=NIFI-2961)](https://travis-ci.org/HandOfGod94/nifi)
--- End diff --

This should be reverted. 


> Create EncryptAttribute processor
> -
>
> Key: NIFI-2961
> URL: https://issues.apache.org/jira/browse/NIFI-2961
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Andy LoPresto
>  Labels: attributes, encryption, security
>
> Similar to {{EncryptContent}}, the {{EncryptAttribute}} processor would allow 
> individual (and multiple) flowfile attributes to be encrypted (either 
> in-place or to a new attribute key) with various encryption algorithms (AES, 
> RSA, PBE, and PGP). 
> Specific compatibility with the {{OpenSSL EVP_BytesToKey}}, {{PBKDF2}}, 
> {{scrypt}}, and {{bcrypt}} key derivation functions should be included. 
> The processor should provide the boolean option to encrypt or decrypt (only 
> one operation per instance of the processor). The processor should also allow 
> Base64 encoding (aka ASCII armor) for the encrypted attributes to prevent 
> byte escaping/data loss. 
> If [dangerous processor 
> annotations|https://cwiki.apache.org/confluence/display/NIFI/Security+Feature+Roadmap]
>  are introduced, this processor should be marked as such and the 
> corresponding attribute protection (i.e. provenance before/after, etc.) 
> should be applied. 
> Originally requested in this [Stack Overflow 
> question|https://stackoverflow.com/questions/40294945/nifi-encrypt-json].  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724322#comment-15724322
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010488
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724321#comment-15724321
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90929270
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724329#comment-15724329
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90783328
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724324#comment-15724324
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010258
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEncryptAttributes.java
 ---
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.security.Security;
+import java.util.Map;
+
+public class TestEncryptAttributes {
+
+private static final Logger logger = 
LoggerFactory.getLogger(TestEncryptAttributes.class);
+
+// Initialize some common property values which will be used for 
setting up processor
+private static final EncryptionMethod[] ENCRYPTION_METHODS = 
EncryptionMethod.values();
+final String RAW_HEX_KEY= "abababababababababababababababab";
+private static final String PRIVATE_KEYRING = 
"src/test/resources/TestEncryptContent/secring.gpg";
+private static final String PUBLIC_KEYRING = 
"src/test/resources/TestEncryptContent/pubring.gpg";
+private static final String PRIVATE_KEYRING_PASSPHRASE = "PASSWORD";
+private static final String FILENAME_ATTR_KEY = 
CoreAttributes.FILENAME.key();
+private static final String UUID_ATTR_KEY = CoreAttributes.UUID.key();
+
+
+@Before
+public void setUp() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+
+@Test
+public void testRoundTrip() {
+final TestRunner testRunner = TestRunners.newTestRunner(new 
EncryptAttributes());
+
+for (final EncryptionMethod encryptionMethod : ENCRYPTION_METHODS) 
{
+if (encryptionMethod.isUnlimitedStrength())
+continue;
+if (encryptionMethod.isKeyedCipher()){
+testRunner.setProperty(EncryptAttributes.RAW_KEY_HEX, 
RAW_HEX_KEY);
+
testRunner.setProperty(EncryptAttributes.KEY_DERIVATION_FUNCTION, 
KeyDerivationFunction.NONE.name());
+} else {
+testRunner.setProperty(EncryptAttributes.PASSWORD, 
"short");
+
testRunner.setProperty(EncryptAttributes.KEY_DERIVATION_FUNCTION, 
KeyDerivationFunction.OPENSSL_EVP_BYTES_TO_KEY.name());
+
testRunner.setProperty(EncryptAttributes.ALLOW_WEAK_CRYPTO, 
EncryptAttributes.WEAK_CRYPTO_ALLOWED_NAME);
+}
+
+logger.info("Attempting {}", encryptionMethod.name());
+testRunner.setProperty(EncryptAttributes.ENCRYPTION_ALGORITHM, 
encryptionMethod.name());
+testRunner.setProperty(EncryptAttributes.MODE, 
EncryptAttributes.ENCRYPT_MODE);
+
+//create FlowFile and pass it to processor
+ProcessSession session = 
testRunner.getProcessSessionFactory().createSession();
+FlowFile ff = session.create();
--- End diff --

The only attribute generated here that will be encrypted is `path`. Why is 
`path` not treated as a core attribute like `filename` and `uuid`?


> Create EncryptAttribute processor
> -
>
>

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90783308
  
--- Diff: README.md ---
@@ -12,7 +12,7 @@
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
-# Apache NiFi [![Build 
Status](https://travis-ci.org/apache/nifi.svg?branch=master)](https://travis-ci.org/apache/nifi)
+# Apache NiFi [![Build 
Status](https://travis-ci.org/HandOfGod94/nifi.svg?branch=NIFI-2961)](https://travis-ci.org/HandOfGod94/nifi)
--- End diff --

This should be reverted. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724330#comment-15724330
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010436
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[jira] [Commented] (NIFI-2961) Create EncryptAttribute processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724318#comment-15724318
 ] 

ASF GitHub Bot commented on NIFI-2961:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90928984
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String 

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010258
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEncryptAttributes.java
 ---
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.security.Security;
+import java.util.Map;
+
+public class TestEncryptAttributes {
+
+private static final Logger logger = 
LoggerFactory.getLogger(TestEncryptAttributes.class);
+
+// Initialize some common property values which will be used for 
setting up processor
+private static final EncryptionMethod[] ENCRYPTION_METHODS = 
EncryptionMethod.values();
+final String RAW_HEX_KEY= "abababababababababababababababab";
+private static final String PRIVATE_KEYRING = 
"src/test/resources/TestEncryptContent/secring.gpg";
+private static final String PUBLIC_KEYRING = 
"src/test/resources/TestEncryptContent/pubring.gpg";
+private static final String PRIVATE_KEYRING_PASSPHRASE = "PASSWORD";
+private static final String FILENAME_ATTR_KEY = 
CoreAttributes.FILENAME.key();
+private static final String UUID_ATTR_KEY = CoreAttributes.UUID.key();
+
+
+@Before
+public void setUp() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+
+@Test
+public void testRoundTrip() {
+final TestRunner testRunner = TestRunners.newTestRunner(new 
EncryptAttributes());
+
+for (final EncryptionMethod encryptionMethod : ENCRYPTION_METHODS) 
{
+if (encryptionMethod.isUnlimitedStrength())
+continue;
+if (encryptionMethod.isKeyedCipher()){
+testRunner.setProperty(EncryptAttributes.RAW_KEY_HEX, 
RAW_HEX_KEY);
+
testRunner.setProperty(EncryptAttributes.KEY_DERIVATION_FUNCTION, 
KeyDerivationFunction.NONE.name());
+} else {
+testRunner.setProperty(EncryptAttributes.PASSWORD, 
"short");
+
testRunner.setProperty(EncryptAttributes.KEY_DERIVATION_FUNCTION, 
KeyDerivationFunction.OPENSSL_EVP_BYTES_TO_KEY.name());
+
testRunner.setProperty(EncryptAttributes.ALLOW_WEAK_CRYPTO, 
EncryptAttributes.WEAK_CRYPTO_ALLOWED_NAME);
+}
+
+logger.info("Attempting {}", encryptionMethod.name());
+testRunner.setProperty(EncryptAttributes.ENCRYPTION_ALGORITHM, 
encryptionMethod.name());
+testRunner.setProperty(EncryptAttributes.MODE, 
EncryptAttributes.ENCRYPT_MODE);
+
+//create FlowFile and pass it to processor
+ProcessSession session = 
testRunner.getProcessSessionFactory().createSession();
+FlowFile ff = session.create();
--- End diff --

The only attribute generated here that will be encrypted is `path`. Why is 
`path` not treated as a core attribute like `filename` and `uuid`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with 

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90999588
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91009609
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/resources/docs/org/apache/nifi/processors/standard/EncryptAttributes/additionalDetails.html
 ---
@@ -0,0 +1,30 @@
+
--- End diff --

The message contained here is not relevant to `EncryptAttributes` so this 
file can be removed. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010436
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90928984
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010400
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90929270
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91010488
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91009968
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r91000619
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1294: NIFI-2961 Create EncryptAttribute processor

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1294#discussion_r90783328
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptAttributes.java
 ---
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.crypto.CipherUtility;
+import org.apache.nifi.processors.standard.util.crypto.KeyedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPKeyBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.OpenPGPPasswordBasedEncryptor;
+import 
org.apache.nifi.processors.standard.util.crypto.PasswordBasedEncryptor;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.text.Normalizer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Provides functionality of encrypting attributes with various algorithms.
+ * Note. It'll not modify filename or uuid as they are sensitive and are
+ * internally used by either Algorithm itself or FlowFile repo.
+ */
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"encryption", "decryption", "password", "JCE", "OpenPGP", "PGP", 
"GPG"})
+@CapabilityDescription("Encrypts or Decrypts a FlowFile attributes using 
either symmetric encryption with a password " +
+"and randomly generated salt, or asymmetric encryption using a 
public and secret key.")
+public class EncryptAttributes extends AbstractProcessor {
+
+public static final String ENCRYPT_MODE = "Encrypt";
+public static final String DECRYPT_MODE = "Decrypt";
+
+public static final String WEAK_CRYPTO_ALLOWED_NAME = "allowed";
+public static final String WEAK_CRYPTO_NOT_ALLOWED_NAME = 
"not-allowed";
+
+public static final PropertyDescriptor ATTRIBUTES_TO_ENCRYPT = new 
PropertyDescriptor.Builder()
+

[GitHub] nifi pull request #1293: NIFI-3131 - Qualifying nifi-env in nifi bat scripts...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1293


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3131) run-nifi.bat file fails if run from another directory

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724259#comment-15724259
 ] 

ASF subversion and git services commented on NIFI-3131:
---

Commit 97d2d30423b023abdf9c582d4590ddc8fe41a4d8 in nifi's branch 
refs/heads/master from [~bryanrosan...@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=97d2d30 ]

NIFI-3131 - Qualifying nifi-env in nifi bat scripts, removing popd from toolkit 
bat scripts

This closes #1293

Signed-off-by: Bryan Rosander 


> run-nifi.bat file fails if run from another directory
> -
>
> Key: NIFI-3131
> URL: https://issues.apache.org/jira/browse/NIFI-3131
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Attempting to run run-nifi.bat from a different directory (I tried the parent 
> dir) results in problems resolving relative paths and the failure to properly 
> call the nifi-env.bat.
> {code}
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0>bin\run-nifi.bat
> 'nifi-env.bat' is not recognized as an internal or external command,
> operable program or batch file.
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback.groovy]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback-test.xml]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found 
> resource [logback.xml] at 
> [file:/C:/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/conf/logback.xml]
> 12:28:57,970 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not 
> set
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Setting 
> ReconfigureOnChangeFilter scanning period to 30 seconds
> 12:28:57,985 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will 
> scan for changes in 
> [[C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\logback.xml]] every 
> 30 seconds.
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Adding 
> ReconfigureOnChangeFilter as a turbo filter
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Adding 
> LoggerContextListener of type 
> [ch.qos.logback.classic.jul.LevelChangePropagator] to the object stack
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.jul.LevelChangePropagator@6debcae2 - Propagating DEBUG 
> level on Logger[ROOT] onto the JUL framework
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Starting 
> LoggerContextListener
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> About to instantiate appender of type 
> [ch.qos.logback.core.rolling.RollingFileAppender]
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> Naming appender as [APP_FILE]
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No 
> compression will be used
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use 
> the pattern /nifi-app_%d{-MM-dd_HH}.%i.log for the active file
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - The date pattern 
> is '-MM-dd_HH' from file name pattern 
> '/nifi-app_%d{-MM-dd_HH}.%i.log'.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Roll-over at the 
> top of every hour.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Setting initial 
> period to Wed Nov 30 12:28:58 EST 2016
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - Active log file 
> name: /nifi-app.log
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - File property is 
> set to [/nifi-app.log]
> 12:28:58,173 |-ERROR in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - 
> openFile(/nifi-app.log,true) call failed. java.io.FileNotFoundException: 
> \nifi-app.log (Access is denied)
> at java.io.FileNotFoundException: \nifi-app.log (Access is denied)
> at  at java.io.FileOutputStream.open0(Native Method)
> at  at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at  at java.io.FileOutputStream.(FileOutputStream.java:213)
> at  at 
> ch.qos.logback.core.recovery.ResilientFileOutputStream.(ResilientFileOutputStream.java:28)
> at  at 
> ch.qos.logback.core.FileAppender.openFile(FileAppender.java:148)
> at  at 
> ch.qos.logback.core.FileAppender.start(FileAppender.java:108)
> at  at 
> 

[jira] [Commented] (NIFI-3131) run-nifi.bat file fails if run from another directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724261#comment-15724261
 ] 

ASF GitHub Bot commented on NIFI-3131:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1293


> run-nifi.bat file fails if run from another directory
> -
>
> Key: NIFI-3131
> URL: https://issues.apache.org/jira/browse/NIFI-3131
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Attempting to run run-nifi.bat from a different directory (I tried the parent 
> dir) results in problems resolving relative paths and the failure to properly 
> call the nifi-env.bat.
> {code}
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0>bin\run-nifi.bat
> 'nifi-env.bat' is not recognized as an internal or external command,
> operable program or batch file.
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback.groovy]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback-test.xml]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found 
> resource [logback.xml] at 
> [file:/C:/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/conf/logback.xml]
> 12:28:57,970 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not 
> set
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Setting 
> ReconfigureOnChangeFilter scanning period to 30 seconds
> 12:28:57,985 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will 
> scan for changes in 
> [[C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\logback.xml]] every 
> 30 seconds.
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Adding 
> ReconfigureOnChangeFilter as a turbo filter
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Adding 
> LoggerContextListener of type 
> [ch.qos.logback.classic.jul.LevelChangePropagator] to the object stack
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.jul.LevelChangePropagator@6debcae2 - Propagating DEBUG 
> level on Logger[ROOT] onto the JUL framework
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Starting 
> LoggerContextListener
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> About to instantiate appender of type 
> [ch.qos.logback.core.rolling.RollingFileAppender]
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> Naming appender as [APP_FILE]
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No 
> compression will be used
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use 
> the pattern /nifi-app_%d{-MM-dd_HH}.%i.log for the active file
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - The date pattern 
> is '-MM-dd_HH' from file name pattern 
> '/nifi-app_%d{-MM-dd_HH}.%i.log'.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Roll-over at the 
> top of every hour.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Setting initial 
> period to Wed Nov 30 12:28:58 EST 2016
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - Active log file 
> name: /nifi-app.log
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - File property is 
> set to [/nifi-app.log]
> 12:28:58,173 |-ERROR in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - 
> openFile(/nifi-app.log,true) call failed. java.io.FileNotFoundException: 
> \nifi-app.log (Access is denied)
> at java.io.FileNotFoundException: \nifi-app.log (Access is denied)
> at  at java.io.FileOutputStream.open0(Native Method)
> at  at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at  at java.io.FileOutputStream.(FileOutputStream.java:213)
> at  at 
> ch.qos.logback.core.recovery.ResilientFileOutputStream.(ResilientFileOutputStream.java:28)
> at  at 
> ch.qos.logback.core.FileAppender.openFile(FileAppender.java:148)
> at  at 
> ch.qos.logback.core.FileAppender.start(FileAppender.java:108)
> at  at 
> ch.qos.logback.core.rolling.RollingFileAppender.start(RollingFileAppender.java:86)
> at  at 
> ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:96)
> at  at 
> ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:317)
> at  

[jira] [Commented] (NIFI-3131) run-nifi.bat file fails if run from another directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724228#comment-15724228
 ] 

ASF GitHub Bot commented on NIFI-3131:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1293
  
Merging after review by @jtstorck 


> run-nifi.bat file fails if run from another directory
> -
>
> Key: NIFI-3131
> URL: https://issues.apache.org/jira/browse/NIFI-3131
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Attempting to run run-nifi.bat from a different directory (I tried the parent 
> dir) results in problems resolving relative paths and the failure to properly 
> call the nifi-env.bat.
> {code}
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0>bin\run-nifi.bat
> 'nifi-env.bat' is not recognized as an internal or external command,
> operable program or batch file.
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback.groovy]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback-test.xml]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found 
> resource [logback.xml] at 
> [file:/C:/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/conf/logback.xml]
> 12:28:57,970 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not 
> set
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Setting 
> ReconfigureOnChangeFilter scanning period to 30 seconds
> 12:28:57,985 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will 
> scan for changes in 
> [[C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\logback.xml]] every 
> 30 seconds.
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Adding 
> ReconfigureOnChangeFilter as a turbo filter
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Adding 
> LoggerContextListener of type 
> [ch.qos.logback.classic.jul.LevelChangePropagator] to the object stack
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.jul.LevelChangePropagator@6debcae2 - Propagating DEBUG 
> level on Logger[ROOT] onto the JUL framework
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Starting 
> LoggerContextListener
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> About to instantiate appender of type 
> [ch.qos.logback.core.rolling.RollingFileAppender]
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> Naming appender as [APP_FILE]
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No 
> compression will be used
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use 
> the pattern /nifi-app_%d{-MM-dd_HH}.%i.log for the active file
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - The date pattern 
> is '-MM-dd_HH' from file name pattern 
> '/nifi-app_%d{-MM-dd_HH}.%i.log'.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Roll-over at the 
> top of every hour.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Setting initial 
> period to Wed Nov 30 12:28:58 EST 2016
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - Active log file 
> name: /nifi-app.log
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - File property is 
> set to [/nifi-app.log]
> 12:28:58,173 |-ERROR in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - 
> openFile(/nifi-app.log,true) call failed. java.io.FileNotFoundException: 
> \nifi-app.log (Access is denied)
> at java.io.FileNotFoundException: \nifi-app.log (Access is denied)
> at  at java.io.FileOutputStream.open0(Native Method)
> at  at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at  at java.io.FileOutputStream.(FileOutputStream.java:213)
> at  at 
> ch.qos.logback.core.recovery.ResilientFileOutputStream.(ResilientFileOutputStream.java:28)
> at  at 
> ch.qos.logback.core.FileAppender.openFile(FileAppender.java:148)
> at  at 
> ch.qos.logback.core.FileAppender.start(FileAppender.java:108)
> at  at 
> ch.qos.logback.core.rolling.RollingFileAppender.start(RollingFileAppender.java:86)
> at  at 
> ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:96)
> at  at 
> 

[GitHub] nifi issue #1293: NIFI-3131 - Qualifying nifi-env in nifi bat scripts, remov...

[brosander]

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1293
  
Merging after review by @jtstorck 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3132) sh scripts resolve log directory, some classpaths incorrectly in cygwin

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724222#comment-15724222
 ] 

ASF subversion and git services commented on NIFI-3132:
---

Commit d1f2492dec386228432dc3374d26944c8bef501e in nifi's branch 
refs/heads/master from [~bryanrosan...@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d1f2492 ]

NIFI-3132 - Using cygpath and quoting paths correctly in sh scripts when run in 
cygwin

This closes #1292

Signed-off-by: Bryan Rosander 


> sh scripts resolve log directory, some classpaths incorrectly in cygwin
> ---
>
> Key: NIFI-3132
> URL: https://issues.apache.org/jira/browse/NIFI-3132
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Running in cygwin results in the incorrect directory used for log directory.  
> The other relative paths appear to resolve properly.
> Please see org.apache.nifi.bootstrap.config.log.dir compared to other paths.
> This results in logs being created at:
> C:\cygdrive\c\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> instead of:
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> {code}
> bryan@bryan-win-vm /cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0
> $ bin/nifi.sh run
> Java home: /cygdrive/c/Program Files/Java/jdk1.8.0_92
> NiFi home: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> Bootstrap Config File: 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\bootstrap.conf
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Starting Apache NiFi...
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command Working 
> Directory: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Command: java -classpath 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jcl-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jul-to-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\log4j-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-classic-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-core-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-documentation-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-framework-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-nar-utils-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-properties-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-runtime-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\slf4j-api-1.7.12.jar
>  -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m 
> -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
> -Djava.awt.headless=true -XX:+UseG1GC 
> -Djava.protocol.handler.pkgs=sun.net.www.protocol 
> -Dnifi.properties.file.path=C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf\nifi.properties
>  -Dnifi.bootstrap.listen.port=50031 -Dapp=NiFi 
> -Dorg.apache.nifi.bootstrap.config.log.dir=/cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/logs
>  org.apache.nifi.NiFi
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-3132) sh scripts resolve log directory, some classpaths incorrectly in cygwin

[Bryan Rosander (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Rosander resolved NIFI-3132.
--
Resolution: Fixed

> sh scripts resolve log directory, some classpaths incorrectly in cygwin
> ---
>
> Key: NIFI-3132
> URL: https://issues.apache.org/jira/browse/NIFI-3132
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Running in cygwin results in the incorrect directory used for log directory.  
> The other relative paths appear to resolve properly.
> Please see org.apache.nifi.bootstrap.config.log.dir compared to other paths.
> This results in logs being created at:
> C:\cygdrive\c\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> instead of:
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> {code}
> bryan@bryan-win-vm /cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0
> $ bin/nifi.sh run
> Java home: /cygdrive/c/Program Files/Java/jdk1.8.0_92
> NiFi home: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> Bootstrap Config File: 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\bootstrap.conf
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Starting Apache NiFi...
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command Working 
> Directory: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Command: java -classpath 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jcl-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jul-to-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\log4j-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-classic-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-core-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-documentation-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-framework-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-nar-utils-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-properties-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-runtime-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\slf4j-api-1.7.12.jar
>  -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m 
> -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
> -Djava.awt.headless=true -XX:+UseG1GC 
> -Djava.protocol.handler.pkgs=sun.net.www.protocol 
> -Dnifi.properties.file.path=C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf\nifi.properties
>  -Dnifi.bootstrap.listen.port=50031 -Dapp=NiFi 
> -Dorg.apache.nifi.bootstrap.config.log.dir=/cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/logs
>  org.apache.nifi.NiFi
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3132) sh scripts resolve log directory, some classpaths incorrectly in cygwin

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724223#comment-15724223
 ] 

ASF GitHub Bot commented on NIFI-3132:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1292


> sh scripts resolve log directory, some classpaths incorrectly in cygwin
> ---
>
> Key: NIFI-3132
> URL: https://issues.apache.org/jira/browse/NIFI-3132
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Running in cygwin results in the incorrect directory used for log directory.  
> The other relative paths appear to resolve properly.
> Please see org.apache.nifi.bootstrap.config.log.dir compared to other paths.
> This results in logs being created at:
> C:\cygdrive\c\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> instead of:
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> {code}
> bryan@bryan-win-vm /cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0
> $ bin/nifi.sh run
> Java home: /cygdrive/c/Program Files/Java/jdk1.8.0_92
> NiFi home: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> Bootstrap Config File: 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\bootstrap.conf
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Starting Apache NiFi...
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command Working 
> Directory: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Command: java -classpath 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jcl-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jul-to-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\log4j-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-classic-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-core-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-documentation-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-framework-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-nar-utils-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-properties-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-runtime-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\slf4j-api-1.7.12.jar
>  -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m 
> -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
> -Djava.awt.headless=true -XX:+UseG1GC 
> -Djava.protocol.handler.pkgs=sun.net.www.protocol 
> -Dnifi.properties.file.path=C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf\nifi.properties
>  -Dnifi.bootstrap.listen.port=50031 -Dapp=NiFi 
> -Dorg.apache.nifi.bootstrap.config.log.dir=/cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/logs
>  org.apache.nifi.NiFi
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1292: NIFI-3132 - Using cygpath and quoting paths correct...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1292


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck resolved NIFI-3150.
---
Resolution: Fixed

> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724176#comment-15724176
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1301
  
Merge commit: 
https://github.com/apache/nifi/commit/78de10dec0127598f8ac7f7048cec3aecb6b


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724175#comment-15724175
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1301
  
@jtstorck thanks for closing, I forgot the "this closes line"


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724172#comment-15724172
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1301
  
Was merged, closing


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1301: NIFI-3150 Added logic to wait for the zk client to connect...

[brosander]

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1301
  
Merge commit: 
https://github.com/apache/nifi/commit/78de10dec0127598f8ac7f7048cec3aecb6b


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724173#comment-15724173
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user jtstorck closed the pull request at:

https://github.com/apache/nifi/pull/1301


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1301: NIFI-3150 Added logic to wait for the zk client to ...

[jtstorck]

Github user jtstorck closed the pull request at:

https://github.com/apache/nifi/pull/1301


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724167#comment-15724167
 ] 

ASF subversion and git services commented on NIFI-3150:
---

Commit 78de10dec0127598f8ac7f7048cec3aecb6b in nifi's branch 
refs/heads/master from [~jtstorck]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=78de10d ]

NIFI-3150 Added logic to wait for the zk client to connect to the configured 
server

Signed-off-by: Bryan Rosander 


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724165#comment-15724165
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1301
  
@jtstorck makes sense, +1, merging


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1301: NIFI-3150 Added logic to wait for the zk client to connect...

[brosander]

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1301
  
@jtstorck makes sense, +1, merging


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1301: NIFI-3150 Added logic to wait for the zk client to connect...

[jtstorck]

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1301
  
@brosander Those close calls were not in the original contrib for the 
ZooKeeper Migrator.  I will investigate this further and make another 
contribution if it is discovered that there is an issue preventing all nodes 
being read/written.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724085#comment-15724085
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1301
  
@brosander Those close calls were not in the original contrib for the 
ZooKeeper Migrator.  I will investigate this further and make another 
contribution if it is discovered that there is an issue preventing all nodes 
being read/written.


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3132) sh scripts resolve log directory, some classpaths incorrectly in cygwin

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724044#comment-15724044
 ] 

ASF GitHub Bot commented on NIFI-3132:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1292
  
Merging after review by @jtstorck 


> sh scripts resolve log directory, some classpaths incorrectly in cygwin
> ---
>
> Key: NIFI-3132
> URL: https://issues.apache.org/jira/browse/NIFI-3132
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Running in cygwin results in the incorrect directory used for log directory.  
> The other relative paths appear to resolve properly.
> Please see org.apache.nifi.bootstrap.config.log.dir compared to other paths.
> This results in logs being created at:
> C:\cygdrive\c\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> instead of:
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> {code}
> bryan@bryan-win-vm /cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0
> $ bin/nifi.sh run
> Java home: /cygdrive/c/Program Files/Java/jdk1.8.0_92
> NiFi home: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> Bootstrap Config File: 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\bootstrap.conf
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Starting Apache NiFi...
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command Working 
> Directory: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Command: java -classpath 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jcl-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jul-to-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\log4j-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-classic-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-core-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-documentation-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-framework-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-nar-utils-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-properties-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-runtime-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\slf4j-api-1.7.12.jar
>  -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m 
> -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
> -Djava.awt.headless=true -XX:+UseG1GC 
> -Djava.protocol.handler.pkgs=sun.net.www.protocol 
> -Dnifi.properties.file.path=C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf\nifi.properties
>  -Dnifi.bootstrap.listen.port=50031 -Dapp=NiFi 
> -Dorg.apache.nifi.bootstrap.config.log.dir=/cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/logs
>  org.apache.nifi.NiFi
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3155) Remote Group Port UUIDs

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-3155:
-

 Summary: Remote Group Port UUIDs
 Key: NIFI-3155
 URL: https://issues.apache.org/jira/browse/NIFI-3155
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Critical
 Fix For: 1.2.0


Remote Group Ports assume the UUID of the port on the target instance. This is 
critical for retaining which port the connection is associated with. However, 
this is problematic for any flow which contains multiple RPGs pointed to the 
same target instance. Associating the underlying component when only an ID is 
known (ie provenance) is impossible as the UUID is ambiguous.

This issue also exists for self-referencing RPGs but is mitigated with extra 
logic around these troublesome scenarios. For instance, we can differentiate 
the Remote Group Port from the Root Group Port of a self-referencing RPG by 
looking at the component type. However, this isn't possible with multiple RPGs 
referencing the same target instance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1292: NIFI-3132 - Using cygpath and quoting paths correctly in s...

[brosander]

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1292
  
Merging after review by @jtstorck 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723976#comment-15723976
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1301
  
@jtstorck is it safe to remove that? What happens if the main method 
terminates before the reads/writes are finished?


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1301: NIFI-3150 Added logic to wait for the zk client to connect...

[brosander]

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1301
  
@jtstorck is it safe to remove that? What happens if the main method 
terminates before the reads/writes are finished?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (NIFI-3133) Unable to empty queue connected to a RPG

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall resolved NIFI-3133.

Resolution: Fixed

> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723880#comment-15723880
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723876#comment-15723876
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723878#comment-15723878
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723874#comment-15723874
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723885#comment-15723885
 ] 

ASF GitHub Bot commented on NIFI-3133:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1297


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723881#comment-15723881
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723875#comment-15723875
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723883#comment-15723883
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723882#comment-15723882
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723877#comment-15723877
 ] 

ASF subversion and git services commented on NIFI-3133:
---

Commit 2d6bba080f90a45a9f4149f6844f452150ed6bc1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2d6bba0 ]

NIFI-3133: - Ensuring that Remote Group Ports are always authorized according 
to their Remote Process Group.

NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self 
referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's 
handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component 
is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall 


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1297: NIFI-3133: Ensure correct checks against Remote Gro...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1297


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723870#comment-15723870
 ] 

ASF GitHub Bot commented on NIFI-3133:
--

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1297
  
+1

Visually verified code and did a contrib-check build. In a secure 3 node 
cluster ran through a number of tests involving RPGs, Input/Output ports and 
policies. I also ran a standalone instance to verify nothing there regressed. 
Everything worked as expected. 

Thanks @mcgilman I will squash and merge.



> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1297: NIFI-3133: Ensure correct checks against Remote Group Port...

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1297
  
+1

Visually verified code and did a contrib-check build. In a secure 3 node 
cluster ran through a number of tests involving RPGs, Input/Output ports and 
policies. I also ran a standalone instance to verify nothing there regressed. 
Everything worked as expected. 

Thanks @mcgilman I will squash and merge.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3154) Connection Config and Connection Details dialogs .label elements text overflow

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-3154:
--
Attachment: connection dialog text label missing ellipsis.png

> Connection Config and Connection Details dialogs .label elements text overflow
> --
>
> Key: NIFI-3154
> URL: https://issues.apache.org/jira/browse/NIFI-3154
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.1.1
>Reporter: Scott Aslan
>Assignee: Scott Aslan
>Priority: Minor
> Fix For: 1.2.0
>
> Attachments: connection dialog text label missing ellipsis.png
>
>
> The Processor name label text does not display ellipsis when sufficiently 
> long.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3154) Connection Config and Connection Details dialogs .label elements text overflow

[Scott Aslan (JIRA)]

Scott Aslan created NIFI-3154:
-

 Summary: Connection Config and Connection Details dialogs .label 
elements text overflow
 Key: NIFI-3154
 URL: https://issues.apache.org/jira/browse/NIFI-3154
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Affects Versions: 1.1.1
Reporter: Scott Aslan
Assignee: Scott Aslan
Priority: Minor
 Fix For: 1.2.0


The Processor name label text does not display ellipsis when sufficiently long.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3132) sh scripts resolve log directory, some classpaths incorrectly in cygwin

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723806#comment-15723806
 ] 

ASF GitHub Bot commented on NIFI-3132:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1292
  
+1
Built under Windows 7, and with a base Cygwin install, was able to run NiFi 
from outside of the bin dir:

`/cygdrive/c/Users/Jeff/nifi-dev/nifi/nifi-assembly/target/nifi-1.2.0-SNAPSHOT-bin/nifi-1.2.0-SNAPSHOT
$ bin/nifi.sh run`

Verified that the log directory location was correct:

`-Dorg.apache.nifi.bootstrap.config.log.dir=C:\Users\Jeff\nifi-dev\nifi\nifi-assembly\target\nifi-1.2.0-SNAPSHOT-bin\nifi-1.2.0-SNAPSHOT\logs`


> sh scripts resolve log directory, some classpaths incorrectly in cygwin
> ---
>
> Key: NIFI-3132
> URL: https://issues.apache.org/jira/browse/NIFI-3132
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Running in cygwin results in the incorrect directory used for log directory.  
> The other relative paths appear to resolve properly.
> Please see org.apache.nifi.bootstrap.config.log.dir compared to other paths.
> This results in logs being created at:
> C:\cygdrive\c\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> instead of:
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\logs
> {code}
> bryan@bryan-win-vm /cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0
> $ bin/nifi.sh run
> Java home: /cygdrive/c/Program Files/Java/jdk1.8.0_92
> NiFi home: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> Bootstrap Config File: 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\bootstrap.conf
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Starting Apache NiFi...
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command Working 
> Directory: C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0
> 2016-11-30 09:57:54,528 INFO [main] org.apache.nifi.bootstrap.Command 
> Command: java -classpath 
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jcl-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\jul-to-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\log4j-over-slf4j-1.7.12.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-classic-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\logback-core-1.1.3.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-documentation-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-framework-api-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-nar-utils-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-properties-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\nifi-runtime-1.1.0.jar;C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\lib\slf4j-api-1.7.12.jar
>  -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m 
> -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
> -Djava.awt.headless=true -XX:+UseG1GC 
> -Djava.protocol.handler.pkgs=sun.net.www.protocol 
> -Dnifi.properties.file.path=C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\.\conf\nifi.properties
>  -Dnifi.bootstrap.listen.port=50031 -Dapp=NiFi 
> -Dorg.apache.nifi.bootstrap.config.log.dir=/cygdrive/c/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/logs
>  org.apache.nifi.NiFi
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1292: NIFI-3132 - Using cygpath and quoting paths correctly in s...

[jtstorck]

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1292
  
+1
Built under Windows 7, and with a base Cygwin install, was able to run NiFi 
from outside of the bin dir:

`/cygdrive/c/Users/Jeff/nifi-dev/nifi/nifi-assembly/target/nifi-1.2.0-SNAPSHOT-bin/nifi-1.2.0-SNAPSHOT
$ bin/nifi.sh run`

Verified that the log directory location was correct:

`-Dorg.apache.nifi.bootstrap.config.log.dir=C:\Users\Jeff\nifi-dev\nifi\nifi-assembly\target\nifi-1.2.0-SNAPSHOT-bin\nifi-1.2.0-SNAPSHOT\logs`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3131) run-nifi.bat file fails if run from another directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723647#comment-15723647
 ] 

ASF GitHub Bot commented on NIFI-3131:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1293
  
+1
Built and ran on Windows 7, and was able to start NiFi with run-nifi.bat 
from outside of the bin directory. 


> run-nifi.bat file fails if run from another directory
> -
>
> Key: NIFI-3131
> URL: https://issues.apache.org/jira/browse/NIFI-3131
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>Priority: Minor
>
> Attempting to run run-nifi.bat from a different directory (I tried the parent 
> dir) results in problems resolving relative paths and the failure to properly 
> call the nifi-env.bat.
> {code}
> C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0>bin\run-nifi.bat
> 'nifi-env.bat' is not recognized as an internal or external command,
> operable program or batch file.
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback.groovy]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could 
> NOT find resource [logback-test.xml]
> 12:28:57,861 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found 
> resource [logback.xml] at 
> [file:/C:/Users/bryan/Downloads/nifi-1.1.0-bin/nifi-1.1.0/conf/logback.xml]
> 12:28:57,970 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not 
> set
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Setting 
> ReconfigureOnChangeFilter scanning period to 30 seconds
> 12:28:57,985 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will 
> scan for changes in 
> [[C:\Users\bryan\Downloads\nifi-1.1.0-bin\nifi-1.1.0\conf\logback.xml]] every 
> 30 seconds.
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.ConfigurationAction - Adding 
> ReconfigureOnChangeFilter as a turbo filter
> 12:28:57,985 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Adding 
> LoggerContextListener of type 
> [ch.qos.logback.classic.jul.LevelChangePropagator] to the object stack
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.jul.LevelChangePropagator@6debcae2 - Propagating DEBUG 
> level on Logger[ROOT] onto the JUL framework
> 12:28:58,017 |-INFO in 
> ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Starting 
> LoggerContextListener
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> About to instantiate appender of type 
> [ch.qos.logback.core.rolling.RollingFileAppender]
> 12:28:58,017 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - 
> Naming appender as [APP_FILE]
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No 
> compression will be used
> 12:28:58,126 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use 
> the pattern /nifi-app_%d{-MM-dd_HH}.%i.log for the active file
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - The date pattern 
> is '-MM-dd_HH' from file name pattern 
> '/nifi-app_%d{-MM-dd_HH}.%i.log'.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Roll-over at the 
> top of every hour.
> 12:28:58,126 |-INFO in 
> ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ba23b66 - Setting initial 
> period to Wed Nov 30 12:28:58 EST 2016
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - Active log file 
> name: /nifi-app.log
> 12:28:58,173 |-INFO in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - File property is 
> set to [/nifi-app.log]
> 12:28:58,173 |-ERROR in 
> ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - 
> openFile(/nifi-app.log,true) call failed. java.io.FileNotFoundException: 
> \nifi-app.log (Access is denied)
> at java.io.FileNotFoundException: \nifi-app.log (Access is denied)
> at  at java.io.FileOutputStream.open0(Native Method)
> at  at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at  at java.io.FileOutputStream.(FileOutputStream.java:213)
> at  at 
> ch.qos.logback.core.recovery.ResilientFileOutputStream.(ResilientFileOutputStream.java:28)
> at  at 
> ch.qos.logback.core.FileAppender.openFile(FileAppender.java:148)
> at  at 
> ch.qos.logback.core.FileAppender.start(FileAppender.java:108)
> at  at 
> ch.qos.logback.core.rolling.RollingFileAppender.start(RollingFileAppender.java:86)
> at  at 
> 

[GitHub] nifi issue #1293: NIFI-3131 - Qualifying nifi-env in nifi bat scripts, remov...

[jtstorck]

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1293
  
+1
Built and ran on Windows 7, and was able to start NiFi with run-nifi.bat 
from outside of the bin directory. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723595#comment-15723595
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90972151
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +287,29 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+if 
(watchedEvent.getType().equals(Watcher.Event.EventType.None) && 
watchedEvent.getState().equals(Watcher.Event.KeeperState.SyncConnected)) {
+connectionLatch.countDown();
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new IOException(String.format("interrupted while waiting 
for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
+}
+
+if (!connected) {
+throw new IOException(String.format("unable to connect to %s, 
state is %s", zooKeeperEndpointConfig, zooKeeper.getState()));
--- End diff --

I don't think it's necessary, since the migrator will exit if a connection 
with ZK cannot be established, but I will add it for the sake of explicit 
cleanup.


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1301: NIFI-3150 Added logic to wait for the zk client to ...

[jtstorck]

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90972151
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +287,29 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+if 
(watchedEvent.getType().equals(Watcher.Event.EventType.None) && 
watchedEvent.getState().equals(Watcher.Event.KeeperState.SyncConnected)) {
+connectionLatch.countDown();
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new IOException(String.format("interrupted while waiting 
for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
+}
+
+if (!connected) {
+throw new IOException(String.format("unable to connect to %s, 
state is %s", zooKeeperEndpointConfig, zooKeeper.getState()));
--- End diff --

I don't think it's necessary, since the migrator will exit if a connection 
with ZK cannot be established, but I will add it for the sake of explicit 
cleanup.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723557#comment-15723557
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90969885
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +287,29 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+if 
(watchedEvent.getType().equals(Watcher.Event.EventType.None) && 
watchedEvent.getState().equals(Watcher.Event.KeeperState.SyncConnected)) {
+connectionLatch.countDown();
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new IOException(String.format("interrupted while waiting 
for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
+}
+
+if (!connected) {
+throw new IOException(String.format("unable to connect to %s, 
state is %s", zooKeeperEndpointConfig, zooKeeper.getState()));
--- End diff --

Should we still try to close the zk connection to free any resources here?


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1301: NIFI-3150 Added logic to wait for the zk client to ...

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90969779
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +287,29 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+if 
(watchedEvent.getType().equals(Watcher.Event.EventType.None) && 
watchedEvent.getState().equals(Watcher.Event.KeeperState.SyncConnected)) {
+connectionLatch.countDown();
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new RuntimeException(String.format("interrupted while 
waiting for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
+}
+
+if (!connected) {
--- End diff --

Should we still try to close the zk connection to free any resources here?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723556#comment-15723556
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90969779
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +287,29 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+if 
(watchedEvent.getType().equals(Watcher.Event.EventType.None) && 
watchedEvent.getState().equals(Watcher.Event.KeeperState.SyncConnected)) {
+connectionLatch.countDown();
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new RuntimeException(String.format("interrupted while 
waiting for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
+}
+
+if (!connected) {
--- End diff --

Should we still try to close the zk connection to free any resources here?


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723553#comment-15723553
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90969482
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +286,40 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+switch (watchedEvent.getType()) {
+case None:
+switch (watchedEvent.getState()) {
+case SyncConnected:
+connectionLatch.countDown();
+break;
+case Expired:
+case AuthFailed:
+case ConnectedReadOnly:
+case SaslAuthenticated:
+case Disconnected:
+break;
+}
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new RuntimeException(String.format("interrupted while 
waiting for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
--- End diff --

However, there are other areas in the code that throw RuntimeExceptions.  
Could address those in another PR if necessary.


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1301: NIFI-3150 Added logic to wait for the zk client to ...

[jtstorck]

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90969482
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +286,40 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+switch (watchedEvent.getType()) {
+case None:
+switch (watchedEvent.getState()) {
+case SyncConnected:
+connectionLatch.countDown();
+break;
+case Expired:
+case AuthFailed:
+case ConnectedReadOnly:
+case SaslAuthenticated:
+case Disconnected:
+break;
+}
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new RuntimeException(String.format("interrupted while 
waiting for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
--- End diff --

However, there are other areas in the code that throw RuntimeExceptions.  
Could address those in another PR if necessary.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723551#comment-15723551
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90969390
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +286,40 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+switch (watchedEvent.getType()) {
+case None:
+switch (watchedEvent.getState()) {
+case SyncConnected:
+connectionLatch.countDown();
+break;
+case Expired:
+case AuthFailed:
+case ConnectedReadOnly:
+case SaslAuthenticated:
+case Disconnected:
+break;
+}
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new RuntimeException(String.format("interrupted while 
waiting for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
--- End diff --

Sure, I can change the RuntimeExceptions to IOExceptions.


> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1301: NIFI-3150 Added logic to wait for the zk client to ...

[jtstorck]

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1301#discussion_r90969390
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/main/java/org/apache/nifi/toolkit/zkmigrator/ZooKeeperMigrator.java
 ---
@@ -284,8 +286,40 @@ private Stat transmitNode(ZooKeeper zooKeeper, 
DataStatAclNode node) {
 }
 
 private ZooKeeper getZooKeeper(ZooKeeperEndpointConfig 
zooKeeperEndpointConfig, AuthMode authMode, byte[] authData) throws IOException 
{
+CountDownLatch connectionLatch = new CountDownLatch(1);
 ZooKeeper zooKeeper = new 
ZooKeeper(zooKeeperEndpointConfig.getConnectString(), 3000, watchedEvent -> {
+if (LOGGER.isDebugEnabled()) {
+LOGGER.debug("ZooKeeper server state changed to {} in {}", 
watchedEvent.getState(), zooKeeperEndpointConfig);
+}
+switch (watchedEvent.getType()) {
+case None:
+switch (watchedEvent.getState()) {
+case SyncConnected:
+connectionLatch.countDown();
+break;
+case Expired:
+case AuthFailed:
+case ConnectedReadOnly:
+case SaslAuthenticated:
+case Disconnected:
+break;
+}
+}
 });
+
+final boolean connected;
+try {
+connected = connectionLatch.await(5, TimeUnit.SECONDS);
+} catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+throw new RuntimeException(String.format("interrupted while 
waiting for ZooKeeper connection to %s", zooKeeperEndpointConfig), e);
--- End diff --

Sure, I can change the RuntimeExceptions to IOExceptions.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723491#comment-15723491
 ] 

ASF GitHub Bot commented on NIFI-3133:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1297#discussion_r90964719
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
 ---
@@ -4034,19 +4034,29 @@ public Authorizable createDataAuthorizable(final 
String componentId) {
 authorizable = new 
DataAuthorizable(connection.getSource());
 }
 } else {
-// authorizable for remote group ports should be the 
remote process group
-if (connectable instanceof RemoteGroupPort) {
-authorizable = new DataAuthorizable(((RemoteGroupPort) 
connectable).getRemoteProcessGroup());
-} else {
-authorizable = new DataAuthorizable(connectable);
-}
+authorizable = new DataAuthorizable(connectable);
 }
 }
 
 return authorizable;
 }
 
 @Override
+public Authorizable createRemoteDataAuthorizable(String 
remoteGroupPortId) {
+final DataAuthorizable authorizable;
+
+final RemoteGroupPort remoteGroupPort = 
getRootGroup().findRemoteGroupPort(remoteGroupPortId);
+if (remoteGroupPort == null) {
+throw new ResourceNotFoundException("The component that 
generated this event is no longer part of the data flow.");
--- End diff --

With the refactoring it is just events. However, there is still one spot 
that is unused that can now be removed. I'll update.


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1297: NIFI-3133: Ensure correct checks against Remote Gro...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1297#discussion_r90965006
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
 ---
@@ -1432,13 +1452,20 @@ public int compare(AttributeDTO a1, AttributeDTO 
a2) {
 private void setComponentDetails(final ProvenanceEventDTO dto) {
 final ProcessGroup root = 
flowController.getGroup(flowController.getRootGroupId());
 
-final Connectable connectable = 
root.findConnectable(dto.getComponentId());
+final Connectable connectable = 
root.findLocalConnectable(dto.getComponentId());
 if (connectable != null) {
 dto.setGroupId(connectable.getProcessGroup().getIdentifier());
 dto.setComponentName(connectable.getName());
 return;
 }
 
+final RemoteGroupPort remoteGroupPort = 
root.findRemoteGroupPort(dto.getComponentId());
+if (connectable != null) {
--- End diff --

Looks like a copy/paste error, this should check "remoteGroupPort"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723496#comment-15723496
 ] 

ASF GitHub Bot commented on NIFI-3133:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1297#discussion_r90965006
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
 ---
@@ -1432,13 +1452,20 @@ public int compare(AttributeDTO a1, AttributeDTO 
a2) {
 private void setComponentDetails(final ProvenanceEventDTO dto) {
 final ProcessGroup root = 
flowController.getGroup(flowController.getRootGroupId());
 
-final Connectable connectable = 
root.findConnectable(dto.getComponentId());
+final Connectable connectable = 
root.findLocalConnectable(dto.getComponentId());
 if (connectable != null) {
 dto.setGroupId(connectable.getProcessGroup().getIdentifier());
 dto.setComponentName(connectable.getName());
 return;
 }
 
+final RemoteGroupPort remoteGroupPort = 
root.findRemoteGroupPort(dto.getComponentId());
+if (connectable != null) {
--- End diff --

Looks like a copy/paste error, this should check "remoteGroupPort"


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1297: NIFI-3133: Ensure correct checks against Remote Gro...

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1297#discussion_r90964719
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
 ---
@@ -4034,19 +4034,29 @@ public Authorizable createDataAuthorizable(final 
String componentId) {
 authorizable = new 
DataAuthorizable(connection.getSource());
 }
 } else {
-// authorizable for remote group ports should be the 
remote process group
-if (connectable instanceof RemoteGroupPort) {
-authorizable = new DataAuthorizable(((RemoteGroupPort) 
connectable).getRemoteProcessGroup());
-} else {
-authorizable = new DataAuthorizable(connectable);
-}
+authorizable = new DataAuthorizable(connectable);
 }
 }
 
 return authorizable;
 }
 
 @Override
+public Authorizable createRemoteDataAuthorizable(String 
remoteGroupPortId) {
+final DataAuthorizable authorizable;
+
+final RemoteGroupPort remoteGroupPort = 
getRootGroup().findRemoteGroupPort(remoteGroupPortId);
+if (remoteGroupPort == null) {
+throw new ResourceNotFoundException("The component that 
generated this event is no longer part of the data flow.");
--- End diff --

With the refactoring it is just events. However, there is still one spot 
that is unused that can now be removed. I'll update.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3150) ZooKeeper Migrator does not wait for connection before attempting to read/write to ZooKeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723489#comment-15723489
 ] 

ASF GitHub Bot commented on NIFI-3150:
--

GitHub user jtstorck opened a pull request:

https://github.com/apache/nifi/pull/1301

NIFI-3150 Added logic to wait for the zk client to connect to the con…

…figured server

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jtstorck/nifi NIFI-3150

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1301.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1301


commit f439f5a42f5d5962ec6e4ccf3beaf6b2189b8374
Author: Jeff Storck 
Date:   2016-12-05T18:19:31Z

NIFI-3150 Added logic to wait for the zk client to connect to the 
configured server




> ZooKeeper Migrator does not wait for connection before attempting to 
> read/write to ZooKeeper
> 
>
> Key: NIFI-3150
> URL: https://issues.apache.org/jira/browse/NIFI-3150
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> While running NiFi in a 3-node cluster, attempting to read or write ZK data 
> to the ZK ensemble results in intermittent connection loss exceptions because 
> the migrator is not waiting for the asynchronous connection method in ZK to 
> report that the client has completely connected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3133) Unable to empty queue connected to a RPG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723479#comment-15723479
 ] 

ASF GitHub Bot commented on NIFI-3133:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1297#discussion_r90964122
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
 ---
@@ -4034,19 +4034,29 @@ public Authorizable createDataAuthorizable(final 
String componentId) {
 authorizable = new 
DataAuthorizable(connection.getSource());
 }
 } else {
-// authorizable for remote group ports should be the 
remote process group
-if (connectable instanceof RemoteGroupPort) {
-authorizable = new DataAuthorizable(((RemoteGroupPort) 
connectable).getRemoteProcessGroup());
-} else {
-authorizable = new DataAuthorizable(connectable);
-}
+authorizable = new DataAuthorizable(connectable);
 }
 }
 
 return authorizable;
 }
 
 @Override
+public Authorizable createRemoteDataAuthorizable(String 
remoteGroupPortId) {
+final DataAuthorizable authorizable;
+
+final RemoteGroupPort remoteGroupPort = 
getRootGroup().findRemoteGroupPort(remoteGroupPortId);
+if (remoteGroupPort == null) {
+throw new ResourceNotFoundException("The component that 
generated this event is no longer part of the data flow.");
--- End diff --

I think there is a bit of copy paste error with this Exception message. My 
understanding is that this method is not limited to an "event".


> Unable to empty queue connected to a RPG
> 
>
> Key: NIFI-3133
> URL: https://issues.apache.org/jira/browse/NIFI-3133
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Critical
> Fix For: 1.2.0
>
>
> Unable to empty a queue in a connection where the source or destination is a 
> Remote Process Group. Appears to be using the policy of the port instead of 
> the RPG which is used for other configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1297: NIFI-3133: Ensure correct checks against Remote Gro...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1297#discussion_r90964122
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
 ---
@@ -4034,19 +4034,29 @@ public Authorizable createDataAuthorizable(final 
String componentId) {
 authorizable = new 
DataAuthorizable(connection.getSource());
 }
 } else {
-// authorizable for remote group ports should be the 
remote process group
-if (connectable instanceof RemoteGroupPort) {
-authorizable = new DataAuthorizable(((RemoteGroupPort) 
connectable).getRemoteProcessGroup());
-} else {
-authorizable = new DataAuthorizable(connectable);
-}
+authorizable = new DataAuthorizable(connectable);
 }
 }
 
 return authorizable;
 }
 
 @Override
+public Authorizable createRemoteDataAuthorizable(String 
remoteGroupPortId) {
+final DataAuthorizable authorizable;
+
+final RemoteGroupPort remoteGroupPort = 
getRootGroup().findRemoteGroupPort(remoteGroupPortId);
+if (remoteGroupPort == null) {
+throw new ResourceNotFoundException("The component that 
generated this event is no longer part of the data flow.");
--- End diff --

I think there is a bit of copy paste error with this Exception message. My 
understanding is that this method is not limited to an "event".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3152) If Provenance Repository runs out of disk space, it may not recover even when disk space is freed up

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723375#comment-15723375
 ] 

ASF GitHub Bot commented on NIFI-3152:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1300


> If Provenance Repository runs out of disk space, it may not recover even when 
> disk space is freed up
> 
>
> Key: NIFI-3152
> URL: https://issues.apache.org/jira/browse/NIFI-3152
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.1.1
>
>
> If we run out of disk space in the provenance repository, we can sometimes 
> get into a situation where the logs show us still waiting for the repo to 
> roll over, even after disk space is freed up. A thread dump shows that the 
> processors are trying to force the repo to rollover. However, the rollover 
> never completes because we can't create an IndexWriter:
> {code}
> "Provenance Repository Rollover Thread-1" Id=128 TIMED_WAITING  on null
>   at java.lang.Thread.sleep(Native Method)
>   at org.apache.lucene.store.Lock.obtain(Lock.java:92)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:104)
>   - waiting on 
> org.apache.nifi.provenance.lucene.SimpleIndexManager@22f9da45
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1711)
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1311)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
>   Number of Locked Synchronizers: 1
>   - java.util.concurrent.ThreadPoolExecutor$Worker@850f87f
> {code}
> The Index Writer is blocking on a lock, waiting to obtain a write lock for 
> the Directory.
> Digging around, I believe the issue is that if we call 
> SimpleIndexManager.returnIndexWriter, it will call IndexWriter.commit(). But 
> if that throws an Exception, we don't properly close the writer. If we are 
> running out of disk space, it is likely that we will throw an Exception on 
> IndexWriter.commit() so this appears to be the root cause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1300: NIFI-3152: Ensure that we always close the IndexWri...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1300


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3152) If Provenance Repository runs out of disk space, it may not recover even when disk space is freed up

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723374#comment-15723374
 ] 

ASF subversion and git services commented on NIFI-3152:
---

Commit fc0d336f36a7ea8d6ba960eead48f4e7e41c070d in nifi's branch 
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=fc0d336 ]

NIFI-3152: Ensure that we always close the IndexWriter when appropriate in 
SimpleIndexManager, even if an IOException is thrown when trying to commit the 
IndexWriter

This closes #1300.

Signed-off-by: Bryan Bende 


> If Provenance Repository runs out of disk space, it may not recover even when 
> disk space is freed up
> 
>
> Key: NIFI-3152
> URL: https://issues.apache.org/jira/browse/NIFI-3152
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.1.1
>
>
> If we run out of disk space in the provenance repository, we can sometimes 
> get into a situation where the logs show us still waiting for the repo to 
> roll over, even after disk space is freed up. A thread dump shows that the 
> processors are trying to force the repo to rollover. However, the rollover 
> never completes because we can't create an IndexWriter:
> {code}
> "Provenance Repository Rollover Thread-1" Id=128 TIMED_WAITING  on null
>   at java.lang.Thread.sleep(Native Method)
>   at org.apache.lucene.store.Lock.obtain(Lock.java:92)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:104)
>   - waiting on 
> org.apache.nifi.provenance.lucene.SimpleIndexManager@22f9da45
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1711)
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1311)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
>   Number of Locked Synchronizers: 1
>   - java.util.concurrent.ThreadPoolExecutor$Worker@850f87f
> {code}
> The Index Writer is blocking on a lock, waiting to obtain a write lock for 
> the Directory.
> Digging around, I believe the issue is that if we call 
> SimpleIndexManager.returnIndexWriter, it will call IndexWriter.commit(). But 
> if that throws an Exception, we don't properly close the writer. If we are 
> running out of disk space, it is likely that we will throw an Exception on 
> IndexWriter.commit() so this appears to be the root cause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3152) If Provenance Repository runs out of disk space, it may not recover even when disk space is freed up

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-3152:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> If Provenance Repository runs out of disk space, it may not recover even when 
> disk space is freed up
> 
>
> Key: NIFI-3152
> URL: https://issues.apache.org/jira/browse/NIFI-3152
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.1.1
>
>
> If we run out of disk space in the provenance repository, we can sometimes 
> get into a situation where the logs show us still waiting for the repo to 
> roll over, even after disk space is freed up. A thread dump shows that the 
> processors are trying to force the repo to rollover. However, the rollover 
> never completes because we can't create an IndexWriter:
> {code}
> "Provenance Repository Rollover Thread-1" Id=128 TIMED_WAITING  on null
>   at java.lang.Thread.sleep(Native Method)
>   at org.apache.lucene.store.Lock.obtain(Lock.java:92)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:104)
>   - waiting on 
> org.apache.nifi.provenance.lucene.SimpleIndexManager@22f9da45
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1711)
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1311)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
>   Number of Locked Synchronizers: 1
>   - java.util.concurrent.ThreadPoolExecutor$Worker@850f87f
> {code}
> The Index Writer is blocking on a lock, waiting to obtain a write lock for 
> the Directory.
> Digging around, I believe the issue is that if we call 
> SimpleIndexManager.returnIndexWriter, it will call IndexWriter.commit(). But 
> if that throws an Exception, we don't properly close the writer. If we are 
> running out of disk space, it is likely that we will throw an Exception on 
> IndexWriter.commit() so this appears to be the root cause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3152) If Provenance Repository runs out of disk space, it may not recover even when disk space is freed up

[Bryan Bende (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723371#comment-15723371
 ] 

Bryan Bende commented on NIFI-3152:
---

Just wanted to document my findings here since my GitHub comment didn't post 
through...

I hard-coded an exception in the old code right after writer.commit() to 
simulate an error, and I let a GenerateFlowFile -> UpdateAttribute run as fast 
as possible... around 418k flow files through, the flow basically froze and 
both processors had an active thread on them, the logs eventually showed:

{code}
2016-12-05 15:50:15,955 ERROR [Provenance Repository Rollover Thread-1] 
o.a.n.p.PersistentProvenanceRepository
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: 
NativeFSLock@/Users/bbende/Projects/bbende-nifi/nifi-assembly/target/nifi-1.2.0-SNAPSHOT-bin/nifi-1.2.0-SNAPSHOT/provenance_repository/index-1480969697000/write.lock
  at org.apache.lucene.store.Lock.obtain(Lock.java:89) ~[na:na]
  at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755) ~[na:na]
  at 
org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:120)
 ~[na:na]
  at 
org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1732)
 ~[na:na]
  at 
org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1323)
 ~[na:na]
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_74]
  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
[na:1.8.0_74]
  at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 [na:1.8.0_74]
  at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 [na:1.8.0_74]
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_74]
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_74]
  at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
2016-12-05 15:50:15,955 WARN [Provenance Repository Rollover Thread-1] 
o.a.n.p.PersistentProvenanceRepository Couldn't merge journals. Will try again. 
journalsToMerge:
{code}

A thread dump also shows the same blocked thread on the index directory as 
described in the JIRA descrption.

Then I waited a few minutes and the processor stats went down to 0 and the 
active threads were still showing so it was clearly stuck.

I retried the same scenario with:
{code}
try {
  writer.commit();
 throw new Exception();
} finally {
  count.close();
}
{code}

And this allowed the rollovers to succeed and the flow continue working well 
past the previous point.

So I'm a +1 on this patch and will merge.

> If Provenance Repository runs out of disk space, it may not recover even when 
> disk space is freed up
> 
>
> Key: NIFI-3152
> URL: https://issues.apache.org/jira/browse/NIFI-3152
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.1.1
>
>
> If we run out of disk space in the provenance repository, we can sometimes 
> get into a situation where the logs show us still waiting for the repo to 
> roll over, even after disk space is freed up. A thread dump shows that the 
> processors are trying to force the repo to rollover. However, the rollover 
> never completes because we can't create an IndexWriter:
> {code}
> "Provenance Repository Rollover Thread-1" Id=128 TIMED_WAITING  on null
>   at java.lang.Thread.sleep(Native Method)
>   at org.apache.lucene.store.Lock.obtain(Lock.java:92)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:104)
>   - waiting on 
> org.apache.nifi.provenance.lucene.SimpleIndexManager@22f9da45
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1711)
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1311)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> 

[jira] [Updated] (NIFI-3106) Mouseover text for Counter reset button is "Connect"

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-3106:
--
Status: Patch Available  (was: In Progress)

> Mouseover text for Counter reset button is "Connect"
> 
>
> Key: NIFI-3106
> URL: https://issues.apache.org/jira/browse/NIFI-3106
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.1.0
>Reporter: Joseph Gresock
>Assignee: Scott Aslan
>Priority: Trivial
> Fix For: 1.2.0
>
>
> This is probably a copy/paste bug from the Connect button on the cluster view.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi-minifi-cpp pull request #26: MINIFI-139 - Resolved OS X 10.12 build fai...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/26


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3152) If Provenance Repository runs out of disk space, it may not recover even when disk space is freed up

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723326#comment-15723326
 ] 

ASF GitHub Bot commented on NIFI-3152:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1300
  
Yep, looks good to me as well. Please merge @bbende. Thanks!


> If Provenance Repository runs out of disk space, it may not recover even when 
> disk space is freed up
> 
>
> Key: NIFI-3152
> URL: https://issues.apache.org/jira/browse/NIFI-3152
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.1.1
>
>
> If we run out of disk space in the provenance repository, we can sometimes 
> get into a situation where the logs show us still waiting for the repo to 
> roll over, even after disk space is freed up. A thread dump shows that the 
> processors are trying to force the repo to rollover. However, the rollover 
> never completes because we can't create an IndexWriter:
> {code}
> "Provenance Repository Rollover Thread-1" Id=128 TIMED_WAITING  on null
>   at java.lang.Thread.sleep(Native Method)
>   at org.apache.lucene.store.Lock.obtain(Lock.java:92)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:104)
>   - waiting on 
> org.apache.nifi.provenance.lucene.SimpleIndexManager@22f9da45
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1711)
>   at 
> org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1311)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
>   Number of Locked Synchronizers: 1
>   - java.util.concurrent.ThreadPoolExecutor$Worker@850f87f
> {code}
> The Index Writer is blocking on a lock, waiting to obtain a write lock for 
> the Directory.
> Digging around, I believe the issue is that if we call 
> SimpleIndexManager.returnIndexWriter, it will call IndexWriter.commit(). But 
> if that throws an Exception, we don't properly close the writer. If we are 
> running out of disk space, it is likely that we will throw an Exception on 
> IndexWriter.commit() so this appears to be the root cause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1300: NIFI-3152: Ensure that we always close the IndexWriter whe...

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1300
  
Yep, looks good to me as well. Please merge @bbende. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---