date:20161110


[ 
https://issues.apache.org/jira/browse/NIFI-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15656352#comment-15656352
 ] 

ASF GitHub Bot commented on NIFI-3025:
--

Github user randerzander commented on the issue:

https://github.com/apache/nifi/pull/1207
  
Tests all passed locally for me, also on travis-ci.. maybe the AppVeyor CI 
infrastructure has problems tonight?

I see two unrelated errors:
1. NPM errors during nifi-web build -- unrelated and not sure they're even 
errors? <3 npm
2. PersistentProvenanceRepository tests -- unsure about this one, but 
passed locally for me


> Bump nifi-spark-receiver's jackson version to match Spark 2.0.1
> ---
>
> Key: NIFI-3025
> URL: https://issues.apache.org/jira/browse/NIFI-3025
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Randy Gelhausen
>Priority: Minor
>
> Apache Spark 2.0.1 uses Jackson 2.6.5.
> When trying to use nifi-spark-receiver with Spark Streaming, including the 
> NiFi artifact conflicts with Spark's included Jackson version.
> Bumping that artifact's Jackson version to 2.6.5 fixes this issue



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1207: NIFI-3025: Bump nifi-spark-receiver's jackson version to m...

2016-11-10 Thread randerzander

Github user randerzander commented on the issue:

https://github.com/apache/nifi/pull/1207
  
Tests all passed locally for me, also on travis-ci.. maybe the AppVeyor CI 
infrastructure has problems tonight?

I see two unrelated errors:
1. NPM errors during nifi-web build -- unrelated and not sure they're even 
errors? <3 npm
2. PersistentProvenanceRepository tests -- unsure about this one, but 
passed locally for me


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-3026) S2S initial connection behavior enhancement

2016-11-10 Thread Koji Kawamura (JIRA)

Koji Kawamura created NIFI-3026:
---

 Summary: S2S initial connection behavior enhancement
 Key: NIFI-3026
 URL: https://issues.apache.org/jira/browse/NIFI-3026
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework, Core UI
Reporter: Koji Kawamura
Assignee: Koji Kawamura


s2s client behavior and initial connection improvement is needed.
Current experience is this: I, as a client (e.g. minifi), connect to a nifi 
cluster of e.g. 10 nodes. but i need to specify 1 node URL to establish this 
connection. this node may not be available 100% and go down, in which case my 
initial connection won't work.
Once S2S makes the first connection, it then has a list of all nodes, and can 
check their status. But first connection failure would be a concern if the 
specified URL is somehow not working. Usually for these problems, the client 
should be able to specify multiple urls (according to multiple target cluster 
nodes), comma-separated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15656081#comment-15656081
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
With the new build I'm seeing errors running though the steps 
[here](https://github.com/jtstorck/docker-zk-krb#run-the-zk-migrator-to-read-the-protected-nodes-for-client-from-zookeeper).

```
2016-11-11 04:13:03,655 WARN [main-SendThread(zk-kerberos:2181)] 
org.apache.zookeeper.ClientCnxn Session 0x0 for server null, unexpected error, 
closing socket connection and attempting reconnect
java.lang.NoClassDefFoundError: org/apache/log4j/Logger
at org.apache.zookeeper.Login.(Login.java:44) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:226)
 ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.client.ZooKeeperSaslClient.(ZooKeeperSaslClient.java:131)
 ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:949) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Logger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
~[na:1.8.0_111]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
~[na:1.8.0_111]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
~[na:1.8.0_111]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
~[na:1.8.0_111]
... 5 common frames omitted
2016-11-11 04:13:03,713 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 
Persisting data from source ZooKeeper: 
ZooKeeperEndpointConfig{connectString=zk-kerberos:2181, path=/}
Exception in thread "main" java.lang.RuntimeException: unable to perform 
operation: KeeperErrorCode = ConnectionLoss for /
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:145)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: 
KeeperErrorCode = ConnectionLoss for /
at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1500)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.getNode(ZooKeeperMigrator.java:188)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.readZooKeeper(ZooKeeperMigrator.java:95)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:137)
```

Unfortunately it looks like Zookeeper has [hard log4j 
dependencies](https://github.com/apache/zookeeper/blob/release-3.4.6/src/java/main/org/apache/zookeeper/Login.java#L44).


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a destination ZooKeeper
> * The toolkit will not allow data to be written to the same ZooKeeper from 
> which the source data was obtained.
> * The toolkit will not support reconnecting to ZooKeeper if it is 
> disconnected.  The user can rerun the tool.
> * The toolkit will support ZooKeepers configured with Kerberos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1193: NIFI-2957 ZooKeeper Migration Toolkit

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
With the new build I'm seeing errors running though the steps 
[here](https://github.com/jtstorck/docker-zk-krb#run-the-zk-migrator-to-read-the-protected-nodes-for-client-from-zookeeper).

```
2016-11-11 04:13:03,655 WARN [main-SendThread(zk-kerberos:2181)] 
org.apache.zookeeper.ClientCnxn Session 0x0 for server null, unexpected error, 
closing socket connection and attempting reconnect
java.lang.NoClassDefFoundError: org/apache/log4j/Logger
at org.apache.zookeeper.Login.(Login.java:44) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:226)
 ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.client.ZooKeeperSaslClient.(ZooKeeperSaslClient.java:131)
 ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:949) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Logger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
~[na:1.8.0_111]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
~[na:1.8.0_111]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
~[na:1.8.0_111]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
~[na:1.8.0_111]
... 5 common frames omitted
2016-11-11 04:13:03,713 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 
Persisting data from source ZooKeeper: 
ZooKeeperEndpointConfig{connectString=zk-kerberos:2181, path=/}
Exception in thread "main" java.lang.RuntimeException: unable to perform 
operation: KeeperErrorCode = ConnectionLoss for /
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:145)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: 
KeeperErrorCode = ConnectionLoss for /
at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1500)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.getNode(ZooKeeperMigrator.java:188)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.readZooKeeper(ZooKeeperMigrator.java:95)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:137)
```

Unfortunately it looks like Zookeeper has [hard log4j 
dependencies](https://github.com/apache/zookeeper/blob/release-3.4.6/src/java/main/org/apache/zookeeper/Login.java#L44).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1191: [NIFI-2898] restore ellipsis for processor type, controlle...

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/1191
  
@moranr @mcgilman I have updated this PR to also include a 'fix' to the 
ellipsis plugin for the view state description.

@moranr I think you were seeing a caching issue with the scrollable 
style...anyway, it should be fixed now with the latest update to this PR. Also 
I was able to apply the styles you suggested in the Jira. Be aware, however, 
there is an issue with the spacing you outlined in the Jira. The issue is that 
the ellipsis plugin requires the width and height of the element be set in 
'px'. With the responsive nature of the dialogs I do not believe we can support 
these descriptions as responsive divs that would be necessary to fulfill said 
spacing (i.e. to support a responsive description area div we would need to 
apply position:absolute). This will require a much larger effort and as the 
original ask for NIFI-2898 was to restore the 0.x ellipsis functionality I 
think we could consider the spacing you outlined as part of a follow on Jira 
that will support the scrollable description areas (as discussed in 
https://issues.apache.org/jira/browse/NIFI-2898). Please let me know what you 
think...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2898) Add Processor dialog cuts off processor descriptions


[ 
https://issues.apache.org/jira/browse/NIFI-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15656053#comment-15656053
 ] 

ASF GitHub Bot commented on NIFI-2898:
--

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/1191
  
@moranr @mcgilman I have updated this PR to also include a 'fix' to the 
ellipsis plugin for the view state description.

@moranr I think you were seeing a caching issue with the scrollable 
style...anyway, it should be fixed now with the latest update to this PR. Also 
I was able to apply the styles you suggested in the Jira. Be aware, however, 
there is an issue with the spacing you outlined in the Jira. The issue is that 
the ellipsis plugin requires the width and height of the element be set in 
'px'. With the responsive nature of the dialogs I do not believe we can support 
these descriptions as responsive divs that would be necessary to fulfill said 
spacing (i.e. to support a responsive description area div we would need to 
apply position:absolute). This will require a much larger effort and as the 
original ask for NIFI-2898 was to restore the 0.x ellipsis functionality I 
think we could consider the spacing you outlined as part of a follow on Jira 
that will support the scrollable description areas (as discussed in 
https://issues.apache.org/jira/browse/NIFI-2898). Please let me know what you 
think...


> Add Processor dialog cuts off processor descriptions
> 
>
> Key: NIFI-2898
> URL: https://issues.apache.org/jira/browse/NIFI-2898
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Jeff Storck
>Assignee: Scott Aslan
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: processor-description-scrolling.png
>
>
> In the Add Processor dialog, descriptions for processors like 
> ConvertJSONToSQL get cut off instead of showing a ellipses.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1206: NIFI-2912 Allow custom text in GenerateFlowFile

2016-11-10 Thread alopresto

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/1206
  
@pvillard31 the only issue I encountered was that if a message is provided, 
the `File Size` field is still required, even though it is not used. In the 
custom validate method, can we either switch `File Size` required to `false` if 
`Custom Text` is populated, or provide a default value of `0B` for `File Size`?

Other than that, seems very useful, and I verified the `contrib-check`, 
tests, and build & install process. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2912) Processor to generate a flow file from a string


[ 
https://issues.apache.org/jira/browse/NIFI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655839#comment-15655839
 ] 

ASF GitHub Bot commented on NIFI-2912:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/1206
  
@pvillard31 the only issue I encountered was that if a message is provided, 
the `File Size` field is still required, even though it is not used. In the 
custom validate method, can we either switch `File Size` required to `false` if 
`Custom Text` is populated, or provide a default value of `0B` for `File Size`?

Other than that, seems very useful, and I verified the `contrib-check`, 
tests, and build & install process. 


> Processor to generate a flow file from a string
> ---
>
> Key: NIFI-2912
> URL: https://issues.apache.org/jira/browse/NIFI-2912
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Alessio Palma
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.2.0
>
>
> Currently to build a flow file from a string we need ReplaceText and 
> GenerateFlowFile with a size of 0B. 
> Is here a way to have use only the GenerateFlowFile processor ? 
> This will be handy when you are using templates in workflows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655771#comment-15655771
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87520923
  
--- Diff: 
nifi-commons/nifi-schema-utils/src/main/java/org/apache/nifi/repository/schema/RecordSchema.java
 ---
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.repository.schema;
+
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class RecordSchema {
+private static final String FIELD_NAME = "Field Name";
+private static final String FIELD_TYPE = "Field Type";
+private static final String REPETITION = "Repetition";
+private static final String SUBFIELDS = "SubFields";
+
+private static final String STRING_TYPE = "String";
+private static final String INT_TYPE = "Integer";
+private static final String LONG_TYPE = "Long";
+private static final String SUBFIELD_TYPE = "SubFieldList";
+
+private final List fields;
+
+public RecordSchema(final List fields) {
+this.fields = fields;
+}
+
+public RecordSchema(final RecordField... fields) {
+this(Arrays.asList(fields));
+}
+
+public List getFields() {
+return fields;
+}
+
+public RecordField getField(final String fieldName) {
+return fields.stream()
+.filter(field -> field.getFieldName().equals(fieldName))
+.findFirst()
+.orElse(null);
+}
+
+public void writeTo(final OutputStream out) throws IOException {
+try {
+final DataOutputStream dos = (out instanceof DataOutputStream) 
? (DataOutputStream) out : new DataOutputStream(out);
+
+dos.writeInt(fields.size());
+for (final RecordField field : fields) {
+writeField(field, dos);
+}
+} catch (final IOException ioe) {
+throw new IOException("Unable to write Record Schema to 
stream", ioe);
+}
+}
+
+private void writeField(final RecordField field, final 
DataOutputStream dos) throws IOException {
+dos.writeInt(4);// 4 fields.
--- End diff --

This comment is/was very confusing to me.  (I believe) this is writing the 
4 values defining a field, not actually writing 4 fields.

This goes hand-in-hand with the loop in "readField" (that it's not reading 
multiple fields but is reading the values defining a field).


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655775#comment-15655775
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87510987
  
--- Diff: 
nifi-commons/nifi-write-ahead-log/src/main/java/org/wali/SingletonSerDeFactory.java
 ---
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.wali;
+
+public class SingletonSerDeFactory implements SerDeFactory {
+private final SerDe serde;
+
+public SingletonSerDeFactory(final SerDe serde) {
+this.serde = serde;
+}
+
+@Override
+public SerDe createSerDe(final String encodingName) {
+return serde;
--- End diff --

Ignoring the "encodingName" parameter seems wrong. Any way to check if the 
serde set matches the passed encoding name?


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1202: NIFI-2854: Refactor repositories and swap files to ...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87517252
  
--- Diff: 
nifi-commons/nifi-schema-utils/src/main/java/org/apache/nifi/repository/schema/UnionRecordField.java
 ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.repository.schema;
+
+import java.util.Arrays;
+import java.util.List;
+
+public class UnionRecordField implements RecordField {
+private final String fieldName;
+private final Repetition repetition;
+private final List possibilities;
+
+public UnionRecordField(final String fieldName, final Repetition 
repetition, final RecordField... possibilities) {
+this(fieldName, repetition, Arrays.asList(possibilities));
+}
+
+public UnionRecordField(final String fieldName, final Repetition 
repetition, final List possibilities) {
--- End diff --

ComplexRecordField and SimpleRecordField call "Objects.requireNonNull" on 
their three inputs but "UnionRecordField" and "MapRecordField" do not. Is that 
intended?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655776#comment-15655776
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87486068
  
--- Diff: 
nifi-framework-api/src/main/java/org/apache/nifi/controller/repository/claim/ResourceClaim.java
 ---
@@ -64,4 +64,28 @@
  * @return true if the Resource Claim is in use, 
false otherwise
  */
 boolean isInUse();
+
+
+/**
+ * Provides the natural ordering for ResourceClaim objects. By default 
they are sorted by their id, then container, then section
+ *
+ * @param other other claim
+ * @return x such that x <=1 if this is less than other;
--- End diff --

I believe this should be "-1"


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655773#comment-15655773
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87487455
  
--- Diff: 
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/BufferedInputStream.java
 ---
@@ -16,19 +16,445 @@
  */
 package org.apache.nifi.stream.io;
 
+import java.io.IOException;
 import java.io.InputStream;
 
 /**
  * This class is a slight modification of the BufferedInputStream in the 
java.io package. The modification is that this implementation does not provide 
synchronization on method calls, which means
  * that this class is not suitable for use by multiple threads. However, 
the absence of these synchronized blocks results in potentially much better 
performance.
  */
-public class BufferedInputStream extends java.io.BufferedInputStream {
+public class BufferedInputStream extends InputStream {
--- End diff --

Can this be renamed to explicitly call out that this is not synchronized? 
This is a part of nifi-utils and I could easily see someone using it 
unknowingly because their IDE gives them the option. Or since it is part of 
nifi-utils that makes it public and can't be changed?


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655772#comment-15655772
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87517252
  
--- Diff: 
nifi-commons/nifi-schema-utils/src/main/java/org/apache/nifi/repository/schema/UnionRecordField.java
 ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.repository.schema;
+
+import java.util.Arrays;
+import java.util.List;
+
+public class UnionRecordField implements RecordField {
+private final String fieldName;
+private final Repetition repetition;
+private final List possibilities;
+
+public UnionRecordField(final String fieldName, final Repetition 
repetition, final RecordField... possibilities) {
+this(fieldName, repetition, Arrays.asList(possibilities));
+}
+
+public UnionRecordField(final String fieldName, final Repetition 
repetition, final List possibilities) {
--- End diff --

ComplexRecordField and SimpleRecordField call "Objects.requireNonNull" on 
their three inputs but "UnionRecordField" and "MapRecordField" do not. Is that 
intended?


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655774#comment-15655774
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87517591
  
--- Diff: 
nifi-commons/nifi-schema-utils/src/main/java/org/apache/nifi/repository/schema/Requirement.java
 ---
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.repository.schema;
+
+public enum Requirement {
--- End diff --

I believe this is leftover from design iterations and it's functionality 
has been merged into "Repetition"


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1202: NIFI-2854: Refactor repositories and swap files to ...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87487455
  
--- Diff: 
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/BufferedInputStream.java
 ---
@@ -16,19 +16,445 @@
  */
 package org.apache.nifi.stream.io;
 
+import java.io.IOException;
 import java.io.InputStream;
 
 /**
  * This class is a slight modification of the BufferedInputStream in the 
java.io package. The modification is that this implementation does not provide 
synchronization on method calls, which means
  * that this class is not suitable for use by multiple threads. However, 
the absence of these synchronized blocks results in potentially much better 
performance.
  */
-public class BufferedInputStream extends java.io.BufferedInputStream {
+public class BufferedInputStream extends InputStream {
--- End diff --

Can this be renamed to explicitly call out that this is not synchronized? 
This is a part of nifi-utils and I could easily see someone using it 
unknowingly because their IDE gives them the option. Or since it is part of 
nifi-utils that makes it public and can't be changed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1202: NIFI-2854: Refactor repositories and swap files to ...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87510987
  
--- Diff: 
nifi-commons/nifi-write-ahead-log/src/main/java/org/wali/SingletonSerDeFactory.java
 ---
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.wali;
+
+public class SingletonSerDeFactory implements SerDeFactory {
+private final SerDe serde;
+
+public SingletonSerDeFactory(final SerDe serde) {
+this.serde = serde;
+}
+
+@Override
+public SerDe createSerDe(final String encodingName) {
+return serde;
--- End diff --

Ignoring the "encodingName" parameter seems wrong. Any way to check if the 
serde set matches the passed encoding name?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1202: NIFI-2854: Refactor repositories and swap files to ...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87520923
  
--- Diff: 
nifi-commons/nifi-schema-utils/src/main/java/org/apache/nifi/repository/schema/RecordSchema.java
 ---
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.repository.schema;
+
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class RecordSchema {
+private static final String FIELD_NAME = "Field Name";
+private static final String FIELD_TYPE = "Field Type";
+private static final String REPETITION = "Repetition";
+private static final String SUBFIELDS = "SubFields";
+
+private static final String STRING_TYPE = "String";
+private static final String INT_TYPE = "Integer";
+private static final String LONG_TYPE = "Long";
+private static final String SUBFIELD_TYPE = "SubFieldList";
+
+private final List fields;
+
+public RecordSchema(final List fields) {
+this.fields = fields;
+}
+
+public RecordSchema(final RecordField... fields) {
+this(Arrays.asList(fields));
+}
+
+public List getFields() {
+return fields;
+}
+
+public RecordField getField(final String fieldName) {
+return fields.stream()
+.filter(field -> field.getFieldName().equals(fieldName))
+.findFirst()
+.orElse(null);
+}
+
+public void writeTo(final OutputStream out) throws IOException {
+try {
+final DataOutputStream dos = (out instanceof DataOutputStream) 
? (DataOutputStream) out : new DataOutputStream(out);
+
+dos.writeInt(fields.size());
+for (final RecordField field : fields) {
+writeField(field, dos);
+}
+} catch (final IOException ioe) {
+throw new IOException("Unable to write Record Schema to 
stream", ioe);
+}
+}
+
+private void writeField(final RecordField field, final 
DataOutputStream dos) throws IOException {
+dos.writeInt(4);// 4 fields.
--- End diff --

This comment is/was very confusing to me.  (I believe) this is writing the 
4 values defining a field, not actually writing 4 fields.

This goes hand-in-hand with the loop in "readField" (that it's not reading 
multiple fields but is reading the values defining a field).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1202: NIFI-2854: Refactor repositories and swap files to ...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1202#discussion_r87517591
  
--- Diff: 
nifi-commons/nifi-schema-utils/src/main/java/org/apache/nifi/repository/schema/Requirement.java
 ---
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.repository.schema;
+
+public enum Requirement {
--- End diff --

I believe this is leftover from design iterations and it's functionality 
has been merged into "Repetition"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655694#comment-15655694
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@brosander Updated the logging config to lessen the logging framework's 
output at startup and added a config to the assembly's classpath dir.


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a destination ZooKeeper
> * The toolkit will not allow data to be written to the same ZooKeeper from 
> which the source data was obtained.
> * The toolkit will not support reconnecting to ZooKeeper if it is 
> disconnected.  The user can rerun the tool.
> * The toolkit will support ZooKeepers configured with Kerberos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1193: NIFI-2957 ZooKeeper Migration Toolkit

2016-11-10 Thread jtstorck

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@brosander Updated the logging config to lessen the logging framework's 
output at startup and added a config to the assembly's classpath dir.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2912) Processor to generate a flow file from a string


[ 
https://issues.apache.org/jira/browse/NIFI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655673#comment-15655673
 ] 

ASF GitHub Bot commented on NIFI-2912:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/1206
  
Reviewing...


> Processor to generate a flow file from a string
> ---
>
> Key: NIFI-2912
> URL: https://issues.apache.org/jira/browse/NIFI-2912
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Alessio Palma
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.2.0
>
>
> Currently to build a flow file from a string we need ReplaceText and 
> GenerateFlowFile with a size of 0B. 
> Is here a way to have use only the GenerateFlowFile processor ? 
> This will be handy when you are using templates in workflows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3025) Bump nifi-spark-receiver's jackson version to match Spark 2.0.1


[ 
https://issues.apache.org/jira/browse/NIFI-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655609#comment-15655609
 ] 

ASF GitHub Bot commented on NIFI-3025:
--

GitHub user randerzander opened a pull request:

https://github.com/apache/nifi/pull/1207

NIFI-3025: Bump nifi-spark-receiver's jackson version to match Spark 2.0.1

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

Yes

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

Yes

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

Yes

- [ ] Is your initial contribution a single, squashed commit?

Yes

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?

Yes

- [ ] Have you written or updated unit tests to verify your changes?

N/A

- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 

N/A

- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?

N/A

- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?

N/A

- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

N/A

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

N/A

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/randerzander/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1207.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1207


commit db4d10a95003e01537a666708c2ed23f1cb48068
Author: Randy Gelhausen 
Date:   2016-11-10T23:44:52Z

NiFi-3025: Bump nifi-spark-receiver's jackson version to match Spark 2.0.1




> Bump nifi-spark-receiver's jackson version to match Spark 2.0.1
> ---
>
> Key: NIFI-3025
> URL: https://issues.apache.org/jira/browse/NIFI-3025
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Randy Gelhausen
>Priority: Minor
>
> Apache Spark 2.0.1 uses Jackson 2.6.5.
> When trying to use nifi-spark-receiver with Spark Streaming, including the 
> NiFi artifact conflicts with Spark's included Jackson version.
> Bumping that artifact's Jackson version to 2.6.5 fixes this issue



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1207: NIFI-3025: Bump nifi-spark-receiver's jackson versi...

2016-11-10 Thread randerzander

GitHub user randerzander opened a pull request:

https://github.com/apache/nifi/pull/1207

NIFI-3025: Bump nifi-spark-receiver's jackson version to match Spark 2.0.1

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

Yes

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

Yes

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

Yes

- [ ] Is your initial contribution a single, squashed commit?

Yes

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?

Yes

- [ ] Have you written or updated unit tests to verify your changes?

N/A

- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 

N/A

- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?

N/A

- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?

N/A

- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

N/A

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

N/A

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/randerzander/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1207.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1207


commit db4d10a95003e01537a666708c2ed23f1cb48068
Author: Randy Gelhausen 
Date:   2016-11-10T23:44:52Z

NiFi-3025: Bump nifi-spark-receiver's jackson version to match Spark 2.0.1




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3025) Bump nifi-spark-receiver's jackson version to match Spark 2.0.1

2016-11-10 Thread Randy Gelhausen (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Randy Gelhausen updated NIFI-3025:
--
Affects Version/s: 1.1.0
   1.0.0

> Bump nifi-spark-receiver's jackson version to match Spark 2.0.1
> ---
>
> Key: NIFI-3025
> URL: https://issues.apache.org/jira/browse/NIFI-3025
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Randy Gelhausen
>Priority: Minor
>
> Apache Spark 2.0.1 uses Jackson 2.6.5.
> When trying to use nifi-spark-receiver with Spark Streaming, including the 
> NiFi artifact conflicts with Spark's included Jackson version.
> Bumping that artifact's Jackson version to 2.6.5 fixes this issue



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3025) Bump nifi-spark-receiver's jackson version to match Spark 2.0.1

2016-11-10 Thread Randy Gelhausen (JIRA)

Randy Gelhausen created NIFI-3025:
-

 Summary: Bump nifi-spark-receiver's jackson version to match Spark 
2.0.1
 Key: NIFI-3025
 URL: https://issues.apache.org/jira/browse/NIFI-3025
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Randy Gelhausen
Priority: Minor


Apache Spark 2.0.1 uses Jackson 2.6.5.

When trying to use nifi-spark-receiver with Spark Streaming, including the NiFi 
artifact conflicts with Spark's included Jackson version.

Bumping that artifact's Jackson version to 2.6.5 fixes this issue



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2912) Processor to generate a flow file from a string


 [ 
https://issues.apache.org/jira/browse/NIFI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-2912:
-
Fix Version/s: 1.2.0
   Status: Patch Available  (was: Open)

> Processor to generate a flow file from a string
> ---
>
> Key: NIFI-2912
> URL: https://issues.apache.org/jira/browse/NIFI-2912
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Alessio Palma
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.2.0
>
>
> Currently to build a flow file from a string we need ReplaceText and 
> GenerateFlowFile with a size of 0B. 
> Is here a way to have use only the GenerateFlowFile processor ? 
> This will be handy when you are using templates in workflows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2912) Processor to generate a flow file from a string


 [ 
https://issues.apache.org/jira/browse/NIFI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-2912:
-
   Assignee: Pierre Villard
Component/s: Extensions

> Processor to generate a flow file from a string
> ---
>
> Key: NIFI-2912
> URL: https://issues.apache.org/jira/browse/NIFI-2912
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Alessio Palma
>Assignee: Pierre Villard
>Priority: Minor
>
> Currently to build a flow file from a string we need ReplaceText and 
> GenerateFlowFile with a size of 0B. 
> Is here a way to have use only the GenerateFlowFile processor ? 
> This will be handy when you are using templates in workflows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1206: NIFI-2912 Allow custom text in GenerateFlowFile

2016-11-10 Thread pvillard31

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/1206

NIFI-2912 Allow custom text in GenerateFlowFile



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-2912

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1206.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1206


commit 06df41be1e3f0961ccb5fe392a6127317d3da4f2
Author: Pierre Villard 
Date:   2016-11-10T23:44:06Z

NIFI-2912 Allow custom text in GenerateFlowFile




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2912) Processor to generate a flow file from a string


[ 
https://issues.apache.org/jira/browse/NIFI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1567#comment-1567
 ] 

ASF GitHub Bot commented on NIFI-2912:
--

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/1206

NIFI-2912 Allow custom text in GenerateFlowFile



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-2912

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1206.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1206


commit 06df41be1e3f0961ccb5fe392a6127317d3da4f2
Author: Pierre Villard 
Date:   2016-11-10T23:44:06Z

NIFI-2912 Allow custom text in GenerateFlowFile




> Processor to generate a flow file from a string
> ---
>
> Key: NIFI-2912
> URL: https://issues.apache.org/jira/browse/NIFI-2912
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Alessio Palma
>Priority: Minor
>
> Currently to build a flow file from a string we need ReplaceText and 
> GenerateFlowFile with a size of 0B. 
> Is here a way to have use only the GenerateFlowFile processor ? 
> This will be handy when you are using templates in workflows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655515#comment-15655515
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1193#discussion_r87508185
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/test/resources/logback.groovy 
---
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import ch.qos.logback.classic.encoder.PatternLayoutEncoder
--- End diff --

Yes, I'll look into that.


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a destination ZooKeeper
> * The toolkit will not allow data to be written to the same ZooKeeper from 
> which the source data was obtained.
> * The toolkit will not support reconnecting to ZooKeeper if it is 
> disconnected.  The user can rerun the tool.
> * The toolkit will support ZooKeepers configured with Kerberos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1193: NIFI-2957 ZooKeeper Migration Toolkit

2016-11-10 Thread jtstorck

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1193#discussion_r87508185
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/test/resources/logback.groovy 
---
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import ch.qos.logback.classic.encoder.PatternLayoutEncoder
--- End diff --

Yes, I'll look into that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2996) Processor/Service validation takes exponentially longer to run as the graph grows


[ 
https://issues.apache.org/jira/browse/NIFI-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655470#comment-15655470
 ] 

ASF GitHub Bot commented on NIFI-2996:
--

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/1192
  
Very good points @mcgilman. Just by numbers, there are more processors than 
services/tasks/ports, but I agree they should all act the same.  It's possible 
that only the getValidationErrors() has significant effect on REST request 
performance.  I will do more experiments and update this PR.

I really wanted this PR to have minimal changes, because I think NIFI-950 
(background asynchronous validation when validating **everything** for the UI) 
is a better approach.


> Processor/Service validation takes exponentially longer to run as the graph 
> grows
> -
>
> Key: NIFI-2996
> URL: https://issues.apache.org/jira/browse/NIFI-2996
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Michael Moser
>Assignee: Michael Moser
> Fix For: 1.1.0
>
>
> As you add processors that reference controller services to your NiFi, the 
> validation that occurs during normal UI usage increases dramatically.
> When running in a cluster, I have to increase the nifi.properties 
> nifi.cluster.node.read.timeout well beyond its 5 sec default timeout in order 
> for the UI to work.  Eventually, simple operations in the UI take close to a 
> minute to happen.
> As a test, I created an SSLContextService using certs created with the 
> amazingly useful nifi-toolkit.  I created a DistributedMapCacheClientService 
> that references this SSLContextService.  Then I created 108 
> FetchDistributedMapCache processors that reference the 
> DistributedMapCacheClient service. I used ${hostname(true)} for my 
> FetchDistributedMapCache processor's Cache Entry Identifier property.
> When NiFi is up with no UI connected, during a single "NiFi status" phase I 
> noticed that the SSLContextService was validated 108 times and the EL 
> hostname(true) was evaluated 216 times.  When I connect the UI and go through 
> a normal status refresh cycle, the SSLContextService was validated 432 times 
> and the EL hostname(true) was evaluated 864 times. These validations take a 
> full second on my slowest machine.
> In NiFi 0.x, the expensive REST API call is 
> /nifi-api/controller/controller-services/node.
> In NiFi 1.x, it appears to divide the work among REST API calls to 
> /nifi-api/process-groups/UUID and 
> /nifi-api/flow/process-groups/UUID/controller-services and 
> /nifi-api/flow/status
> Rather than attempting to fix StandardSSLContextService or the EL 
> HostnameEvaluator, I wonder if there was a more generic approach to helping 
> resolve this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1192: NIFI-2996 validate processors only when they are in STOPPE...

2016-11-10 Thread mosermw

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/1192
  
Very good points @mcgilman. Just by numbers, there are more processors than 
services/tasks/ports, but I agree they should all act the same.  It's possible 
that only the getValidationErrors() has significant effect on REST request 
performance.  I will do more experiments and update this PR.

I really wanted this PR to have minimal changes, because I think NIFI-950 
(background asynchronous validation when validating **everything** for the UI) 
is a better approach.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1193: NIFI-2957 ZooKeeper Migration Toolkit

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@jtstorck having both logback and log4j on the classpath is causing slf4j 
to complain:

```
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
SLF4J: Actual binding is of type 
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655358#comment-15655358
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1193#discussion_r87499434
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/test/resources/logback.groovy 
---
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import ch.qos.logback.classic.encoder.PatternLayoutEncoder
--- End diff --

I like the dsl config option here but can we put it in the classpath/ 
folder of the toolkit assembly?  That should get picked up still be editable 
without recompile.


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a destination ZooKeeper
> * The toolkit will not allow data to be written to the same ZooKeeper from 
> which the source data was obtained.
> * The toolkit will not support reconnecting to ZooKeeper if it is 
> disconnected.  The user can rerun the tool.
> * The toolkit will support ZooKeepers configured with Kerberos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1193: NIFI-2957 ZooKeeper Migration Toolkit

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1193#discussion_r87499434
  
--- Diff: 
nifi-toolkit/nifi-toolkit-zookeeper-migrator/src/test/resources/logback.groovy 
---
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import ch.qos.logback.classic.encoder.PatternLayoutEncoder
--- End diff --

I like the dsl config option here but can we put it in the classpath/ 
folder of the toolkit assembly?  That should get picked up still be editable 
without recompile.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #51: MINIFI-36 Refactoring config change notifiers ...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/51#discussion_r87499038
  
--- Diff: 
minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/RunMiNiFi.java 
---
@@ -1600,9 +1622,13 @@ private void restartInstance() throws IOException {
 }
 }
 
-private static void performTransformation(InputStream configIs, String 
configDestinationPath) throws ConfigurationChangeException, IOException {
-try {
-ConfigTransformer.transformConfigFile(configIs, 
configDestinationPath);
+private static ByteBuffer performTransformation(InputStream configIs, 
String configDestinationPath) throws ConfigurationChangeException, IOException {
+try (ByteArrayOutputStream byteArrayOutputStream = new 
ByteArrayOutputStream();
+TeeInputStream teeInputStream = new 
TeeInputStream(configIs, byteArrayOutputStream)) {
+
+ConfigTransformer.transformConfigFile(teeInputStream, 
configDestinationPath);
+
+return 
ByteBuffer.wrap(byteArrayOutputStream.getUnderlyingBuffer());
--- End diff --

Ah good catch! I will change to "toByteArray" and re-test


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655346#comment-15655346
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@jtstorck having both logback and log4j on the classpath is causing slf4j 
to complain:

```
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
SLF4J: Actual binding is of type 
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]
```


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a destination ZooKeeper
> * The toolkit will not allow data to be written to the same ZooKeeper from 
> which the source data was obtained.
> * The toolkit will not support reconnecting to ZooKeeper if it is 
> disconnected.  The user can rerun the tool.
> * The toolkit will support ZooKeepers configured with Kerberos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi-minifi pull request #51: MINIFI-36 Refactoring config change notifiers ...

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/51#discussion_r87498508
  
--- Diff: 
minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/RunMiNiFi.java 
---
@@ -1600,9 +1622,13 @@ private void restartInstance() throws IOException {
 }
 }
 
-private static void performTransformation(InputStream configIs, String 
configDestinationPath) throws ConfigurationChangeException, IOException {
-try {
-ConfigTransformer.transformConfigFile(configIs, 
configDestinationPath);
+private static ByteBuffer performTransformation(InputStream configIs, 
String configDestinationPath) throws ConfigurationChangeException, IOException {
+try (ByteArrayOutputStream byteArrayOutputStream = new 
ByteArrayOutputStream();
+TeeInputStream teeInputStream = new 
TeeInputStream(configIs, byteArrayOutputStream)) {
+
+ConfigTransformer.transformConfigFile(teeInputStream, 
configDestinationPath);
+
+return 
ByteBuffer.wrap(byteArrayOutputStream.getUnderlyingBuffer());
--- End diff --

@JPercivall I think this is what was causing my constant restarting to 
happen.  If the underlying buffer isn't full, it will result in a bunch of null 
values at the end in the transformed config.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1205: UI - Verifying permissions prior to checking Remote Proces...

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/1205
  
Reviewing


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3023) UI - RPG transmission status


 [ 
https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3023:
--
Status: Patch Available  (was: In Progress)

> UI - RPG transmission status
> 
>
> Key: NIFI-3023
> URL: https://issues.apache.org/jira/browse/NIFI-3023
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> Remote Process Group transmission status is currently attempting to access 
> the component configuration. This is currently preventing the UI from loading 
> for a user without permissions. Need to ensure we are only assigning the 
> classes in question when the user has permissions to view the Remote Process 
> Group. 
> {code}
> Uncaught TypeError: Cannot read property 'transmitting' of undefined
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3023) UI - RPG transmission status


[ 
https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655204#comment-15655204
 ] 

Matt Gilman commented on NIFI-3023:
---

Originally referenced the wrong JIRA before posting PR. Here's the PR reference:

https://github.com/apache/nifi/pull/1205

> UI - RPG transmission status
> 
>
> Key: NIFI-3023
> URL: https://issues.apache.org/jira/browse/NIFI-3023
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> Remote Process Group transmission status is currently attempting to access 
> the component configuration. This is currently preventing the UI from loading 
> for a user without permissions. Need to ensure we are only assigning the 
> classes in question when the user has permissions to view the Remote Process 
> Group. 
> {code}
> Uncaught TypeError: Cannot read property 'transmitting' of undefined
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3009) UI - NaN error when backpressure not configured


 [ 
https://issues.apache.org/jira/browse/NIFI-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3009:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> UI - NaN error when backpressure not configured
> ---
>
> Key: NIFI-3009
> URL: https://issues.apache.org/jira/browse/NIFI-3009
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> When backpressure is not configured on a queue, the console shows the 
> following errors:
> {code}
> Error:  attribute width: Expected length, "NaN".
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1205: UI - Verifying permissions prior to checking Remote...

2016-11-10 Thread mcgilman

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1205

UI - Verifying permissions prior to checking Remote Process Group 
transmission status

NIFI-3032:
- Verifying permissions prior to checking Remote Process Group transmission 
status.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-3023

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1205.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1205


commit 573227678cd869787017782870229d2d375fa88d
Author: Matt Gilman 
Date:   2016-11-10T20:21:49Z

NIFI-3032:
- Verifying permissions prior to checking Remote Process Group transmission 
status.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines


[ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655183#comment-15655183
 ] 

ASF GitHub Bot commented on NIFI-3021:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1204


> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Fix For: 1.1.0
>
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1204: [NIFI-3021] remove white-space:no-wrap on view conf...

2016-11-10 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1204


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines


 [ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-3021:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Fix For: 1.1.0
>
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines

2016-11-10 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-3021:
-
Affects Version/s: 1.0.0
Fix Version/s: 1.1.0

> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Fix For: 1.1.0
>
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines


[ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655168#comment-15655168
 ] 

ASF subversion and git services commented on NIFI-3021:
---

Commit 4957b628fbce1b31391a3be390c68f9289ea5615 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4957b62 ]

[NIFI-3021] remove white-space:no-wrap on view configuration properties tables 
textareas

This closes #1204.


> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines


[ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655151#comment-15655151
 ] 

ASF GitHub Bot commented on NIFI-3021:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1204
  
+1 confirmed fix when testing with ExecuteProcessor, merging.


> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1204: [NIFI-3021] remove white-space:no-wrap on view configurati...

2016-11-10 Thread pvillard31

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1204
  
+1 confirmed fix when testing with ExecuteProcessor, merging.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655132#comment-15655132
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1202
  
Here are the failures:

```
Unapproved licenses:
  
/Users/jpercivall/projects/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/hs_err_pid36001.log
  
/Users/jpercivall/projects/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/hs_err_pid83554.log
```


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1202: NIFI-2854: Refactor repositories and swap files to use sch...

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1202
  
Here are the failures:

```
Unapproved licenses:
  
/Users/jpercivall/projects/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/hs_err_pid36001.log
  
/Users/jpercivall/projects/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/hs_err_pid83554.log
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1202: NIFI-2854: Refactor repositories and swap files to use sch...

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1202
  
Reviewing


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655114#comment-15655114
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1202
  
Looks like the PR failed RAT check in travis


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655112#comment-15655112
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1202
  
Reviewing


> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines

2016-11-10 Thread Scott Aslan (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-3021:
--
Status: Patch Available  (was: In Progress)

https://github.com/apache/nifi/pull/1204

> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3024) Encrypted configuration migrator should be able to update sensitive properties key and migrate flow.xml.gz

2016-11-10 Thread Bryan Rosander (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655063#comment-15655063
 ] 

Bryan Rosander commented on NIFI-3024:
--

Initial implementation will read entire flow.xml.gz into memory for 
transformation.  If this becomes a performance issue in the future, we should 
be able to switch to a streaming transformation model as long as input and 
output files are different.

> Encrypted configuration migrator should be able to update sensitive 
> properties key and migrate flow.xml.gz
> --
>
> Key: NIFI-3024
> URL: https://issues.apache.org/jira/browse/NIFI-3024
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration, Tools and Build
>Affects Versions: 1.0.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>  Labels: config, encryption, security, serialization
>
> In order to allow changing of nifi.sensitive.props.key and updating of the 
> flow.xml.gz, the ConfigEncryptionTool should be able to accept a new value 
> for that field and update encrypted values in the flow.xml.gz appropriately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines


[ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655064#comment-15655064
 ] 

ASF GitHub Bot commented on NIFI-3021:
--

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/1204

[NIFI-3021] remove white-space:no-wrap on view configuration properti…

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

…es tables textareas

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi NIFI-3021

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1204.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1204


commit 230fd95dcc86a69c2915a6c1d8c0539369411b40
Author: Scott Aslan 
Date:   2016-11-10T20:27:26Z

[NIFI-3021] remove white-space:no-wrap on view configuration properties 
tables textareas




> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1204: [NIFI-3021] remove white-space:no-wrap on view conf...

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/1204

[NIFI-3021] remove white-space:no-wrap on view configuration propertiâ¦

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

â¦es tables textareas

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi NIFI-3021

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1204.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1204


commit 230fd95dcc86a69c2915a6c1d8c0539369411b40
Author: Scott Aslan 
Date:   2016-11-10T20:27:26Z

[NIFI-3021] remove white-space:no-wrap on view configuration properties 
tables textareas




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-3024) Encrypted configuration migrator should be able to update sensitive properties key and migrate flow.xml.gz

2016-11-10 Thread Bryan Rosander (JIRA)

Bryan Rosander created NIFI-3024:


 Summary: Encrypted configuration migrator should be able to update 
sensitive properties key and migrate flow.xml.gz
 Key: NIFI-3024
 URL: https://issues.apache.org/jira/browse/NIFI-3024
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Configuration, Tools and Build
Affects Versions: 1.0.0
Reporter: Bryan Rosander


In order to allow changing of nifi.sensitive.props.key and updating of the 
flow.xml.gz, the ConfigEncryptionTool should be able to accept a new value for 
that field and update encrypted values in the flow.xml.gz appropriately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-3024) Encrypted configuration migrator should be able to update sensitive properties key and migrate flow.xml.gz

2016-11-10 Thread Bryan Rosander (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Rosander reassigned NIFI-3024:


Assignee: Bryan Rosander

> Encrypted configuration migrator should be able to update sensitive 
> properties key and migrate flow.xml.gz
> --
>
> Key: NIFI-3024
> URL: https://issues.apache.org/jira/browse/NIFI-3024
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration, Tools and Build
>Affects Versions: 1.0.0
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>  Labels: config, encryption, security, serialization
>
> In order to allow changing of nifi.sensitive.props.key and updating of the 
> flow.xml.gz, the ConfigEncryptionTool should be able to accept a new value 
> for that field and update encrypted values in the flow.xml.gz appropriately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines

2016-11-10 Thread Scott Aslan (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655059#comment-15655059
 ] 

Scott Aslan commented on NIFI-3021:
---

It looks like we are applying a css no-wrap style to the textareas. We will 
need to override this for the properties tables view configurations.

> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines

2016-11-10 Thread Scott Aslan (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reassigned NIFI-3021:
-

Assignee: Scott Aslan

> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>Assignee: Scott Aslan
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3023) UI - RPG transmission status


 [ 
https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3023:
--
Description: 
Remote Process Group transmission status is currently attempting to access the 
component configuration. This is currently preventing the UI from loading for a 
user without permissions. Need to ensure we are only assigning the classes in 
question when the user has permissions to view the Remote Process Group. 

{code}
Uncaught TypeError: Cannot read property 'transmitting' of undefined
{code}

  was:Remote Process Group transmission status is currently attempting to 
access the component configuration. This is currently preventing the UI from 
loading for a user without permissions. Need to ensure we are only assigning 
the classes in question when the user has permissions to view the Remote 
Process Group. 


> UI - RPG transmission status
> 
>
> Key: NIFI-3023
> URL: https://issues.apache.org/jira/browse/NIFI-3023
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> Remote Process Group transmission status is currently attempting to access 
> the component configuration. This is currently preventing the UI from loading 
> for a user without permissions. Need to ensure we are only assigning the 
> classes in question when the user has permissions to view the Remote Process 
> Group. 
> {code}
> Uncaught TypeError: Cannot read property 'transmitting' of undefined
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi-minifi pull request #55: MINIFI-136 - Fixing ordering issue in ConfigTr...

2016-11-10 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/55


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3023) UI - RPG transmission status


 [ 
https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3023:
--
Issue Type: Bug  (was: Improvement)

> UI - RPG transmission status
> 
>
> Key: NIFI-3023
> URL: https://issues.apache.org/jira/browse/NIFI-3023
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> Remote Process Group transmission status is currently attempting to access 
> the component configuration. This is currently preventing the UI from loading 
> for a user without permissions. Need to ensure we are only assigning the 
> classes in question when the user has permissions to view the Remote Process 
> Group. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3023) UI - RPG transmission status

Matt Gilman created NIFI-3023:
-

 Summary: UI - RPG transmission status
 Key: NIFI-3023
 URL: https://issues.apache.org/jira/browse/NIFI-3023
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Matt Gilman
Assignee: Matt Gilman
 Fix For: 1.1.0


Remote Process Group transmission status is currently attempting to access the 
component configuration. This is currently preventing the UI from loading for a 
user without permissions. Need to ensure we are only assigning the classes in 
question when the user has permissions to view the Remote Process Group. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2890) Provenance Repository corruption


[ 
https://issues.apache.org/jira/browse/NIFI-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654975#comment-15654975
 ] 

ASF GitHub Bot commented on NIFI-2890:
--

GitHub user jskora opened a pull request:

https://github.com/apache/nifi/pull/1203

NIFI-2890 Provenance Repository Corruption

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X ] Does your PR title start with NIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [X ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

Correct handling of corrupt journal file that was preventing instance 
startup and causing loss of records from uncorrupt files.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jskora/nifi NIFI-2890-0.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1203.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1203


commit bdb00ecd78d16dd33903c3cfd69271444ccf0f37
Author: Joe Skora 
Date:   2016-11-10T19:52:18Z

NIFI-2890 Provenance Repository Corruption - Correct handling of corrupt 
journal file that was preventing instance startup and causing loss of records 
from uncorrupt files. (0.x)




> Provenance Repository corruption
> 
>
> Key: NIFI-2890
> URL: https://issues.apache.org/jira/browse/NIFI-2890
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Brandon DeVries
>Assignee: Joe Skora
>
> It looks as thought there was a ticket to address this\[1\], but it appears 
> to not have completely addressed the issue. On an 0.6.1 instance (that 
> crashed hard), I'm getting a stack trace on startup indicating a corrupt prov 
> repo. The relevant part of the stack trace (unfortunately typed by hand, so 
> forgive the inevitable typos) is:
> {quote}
> ...
> Caused By java.lang.IllegalArgumentException: No enum constant 
> org.apahe.nifi.provenance.ProvenenceEventType.
> at java.lang/.Enum.valueOf...
> at 
> org.apache.nifi.provenance.ProvenanaceEventType.valueOf(ProvenanaceEventType.java:19)
> at
> org.apache.nifi.provenance.StandardRecordReader.nextRecord(StandardRecordReader.java:284)
> at 
> org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1678)
>  \[2\]
> ...
> {quote}
> In 0.x HEAD, this appears to have moved a bit\[3\]. Same is true for 
> 1.x\[4\]. In both 0.x\[5\] and 1.x\[6\], code was added to check for "bad" 
> records coming from the reader, and skip them. However, this appears to only 
> address the issue if the corrupt / bad record is the *first* record coming 
> out of a reader. If it is a subsequent record, the corruption remains.
> \[1\] https://issues.apache.org/jira/browse/NIFI-803
> \[2\] 
> https://github.com/apache/nifi/blob/rel/nifi-0.6.1/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java#L1678
> \[3\] 
>

[GitHub] nifi pull request #1203: NIFI-2890 Provenance Repository Corruption

2016-11-10 Thread jskora

GitHub user jskora opened a pull request:

https://github.com/apache/nifi/pull/1203

NIFI-2890 Provenance Repository Corruption

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X ] Does your PR title start with NIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [X ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

Correct handling of corrupt journal file that was preventing instance 
startup and causing loss of records from uncorrupt files.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jskora/nifi NIFI-2890-0.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1203.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1203


commit bdb00ecd78d16dd33903c3cfd69271444ccf0f37
Author: Joe Skora 
Date:   2016-11-10T19:52:18Z

NIFI-2890 Provenance Repository Corruption - Correct handling of corrupt 
journal file that was preventing instance startup and causing loss of records 
from uncorrupt files. (0.x)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1189: Remove legacy site to site code to ensure port availabilit...

2016-11-10 Thread markap14

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/1189
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1192: NIFI-2996 validate processors only when they are in STOPPE...

2016-11-10 Thread mcgilman

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1192

Thanks for the PR @mosermw! I was just running it and had some follow-up
questions. It looks as though the proposed changes are conditionally validating
the underlying Processor when it is stopped. However, other validation about
the Processor (specifically regarding the incoming and outgoing connections) is
still happening. Comments about these changes are outlined below according to
the modified method.

**isValid**

isValid is used throughout the application to check processor state usually
prior to performing some action or transitioning to some state. I believe in
this method we still need to run all Processor validation logic to ensure we
know if it's valid when invoking it.

When looking into the context of the isValid usage, it does appear that we
can make some changes when getting the Processor status [1]. By checking if the
scheduled state is running prior to checking the processor validity aligns with
the proposed solution and should help reduce the expense of the status calls
identified in the JIRA.

**getValidationErrors**

Regardless of whether the UI is rendering the validation results, they are
still being returned from the Rest Api. I'm wondering if returning a portion of
the validation errors when the Processor is not stopped is confusing. Would it
be more clear if we moved the rest of the validation inside of the conditional
as well? Meaning, when the Processor is not stopped, this method would return
an empty Collection. When the Processor is stopped, this method would run all
Processor validation.

**Additional Comment**

Should these changes also be applied to Ports, Reporting Tasks, and
Controller Services (if disabled)? I understand that not all of these
components cause lengthy validation times, but I'm trying to consider it from a
consistency perspective from a client of the Rest Api. If a Processor only
returns validation errors when it is stopped, I believe this is how all
components should work too.

[1]
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java#L2704

---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios

2016-11-10 Thread Mark Payne (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-2854:
-
Status: Patch Available  (was: Open)

> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to the swap file header may allow some variation 
> there but the variation should only be hints to optimize how they're 
> processed and not change their behavior otherwise. Changes are only permitted 
> during minor version releases.
> * Provenance repository changes are only permitted during minor version 
> releases.  These changes may include adding or removing fields from existing 
> event types.  If a field is considered required it must always be considered 
> required.  If a field is removed then it must not be a required field and 
> there must be a sensible default an older version could use if that value is 
> not found in new data once rolled back.  New event types may be added.  
> Fields or event types not known to older version, if seen after a rollback, 
> will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios


[ 
https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654799#comment-15654799
 ] 

ASF GitHub Bot commented on NIFI-2854:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/1202

NIFI-2854: Refactor repositories and swap files to use schema-based s…

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

…erialization so that nifi can be rolled back to a previous version after 
an upgrade.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2854

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1202


commit 5167505596e505ac89ed1fe975388ad2f35a963e
Author: Mark Payne 
Date:   2016-10-04T13:38:14Z

NIFI-2854: Refactor repositories and swap files to use schema-based 
serialization so that nifi can be rolled back to a previous version after an 
upgrade.




> Enable repositories to support upgrades and rollback in well defined scenarios
> --
>
> Key: NIFI-2854
> URL: https://issues.apache.org/jira/browse/NIFI-2854
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very 
> important roll in NiFi's ability to be safely upgraded and rolled back.  We 
> need to have well documented behaviors, designs, and version adherence so 
> that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance 
> to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which 
> will break forward/backward compatibility and specifically this means that 
> things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they 
> should be reserved for major releases only and should include a utility to 
> help users with pre-existing data convert from some older format to the newer 
> format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle 
> in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to 
> existing write ahead log record types but no fields can be removed nor can 
> any new types be added.  Once a field is considered required it must remain 
> required.  Changes may only be made across minor version changes - not 
> incremental.
> * Swap File storage should follow very similar rules to the flow file 
> repository.  Adding a schema to

[GitHub] nifi pull request #1202: NIFI-2854: Refactor repositories and swap files to ...

2016-11-10 Thread markap14

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/1202

NIFI-2854: Refactor repositories and swap files to use schema-based sâ¦

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

â¦erialization so that nifi can be rolled back to a previous version after 
an upgrade.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2854

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1202


commit 5167505596e505ac89ed1fe975388ad2f35a963e
Author: Mark Payne 
Date:   2016-10-04T13:38:14Z

NIFI-2854: Refactor repositories and swap files to use schema-based 
serialization so that nifi can be rolled back to a previous version after an 
upgrade.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3014) GetSNMP: add textual oid attribute to flowfile


[ 
https://issues.apache.org/jira/browse/NIFI-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654774#comment-15654774
 ] 

ASF GitHub Bot commented on NIFI-3014:
--

Github user joaohf commented on the issue:

https://github.com/apache/nifi/pull/1196
  
Yes, you are right. I should document this.


> GetSNMP: add textual oid attribute to flowfile
> --
>
> Key: NIFI-3014
> URL: https://issues.apache.org/jira/browse/NIFI-3014
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.1.0
>Reporter: João Henrique Ferreira de Freitas
>Priority: Minor
>
> Sometimes is good to have OID textual descripton. So we could easily to 
> create a JSON document or whatever you like. The final flowfile attribute 
> should be: snmp$textualOid



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1196: NIFI-3014 GetSNMP add textual oid attribute to flowfile

2016-11-10 Thread joaohf

Github user joaohf commented on the issue:

https://github.com/apache/nifi/pull/1196
  
Yes, you are right. I should document this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-3022) DROP Provenance Events is not shown when user right-clicks on connection and chooses to Empty queue

2016-11-10 Thread Mark Payne (JIRA)

Mark Payne created NIFI-3022:


 Summary: DROP Provenance Events is not shown when user 
right-clicks on connection and chooses to Empty queue
 Key: NIFI-3022
 URL: https://issues.apache.org/jira/browse/NIFI-3022
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.0.0
Reporter: Mark Payne
 Fix For: 1.1.0


I dropped the FlowFiles from a FlowFile Queue and then was unable to see the 
Provenance Events. It appears that the events are created and read from the 
repository but are not being authorized properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2890) Provenance Repository corruption


[ 
https://issues.apache.org/jira/browse/NIFI-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654762#comment-15654762
 ] 

ASF GitHub Bot commented on NIFI-2890:
--

GitHub user jskora opened a pull request:

https://github.com/apache/nifi/pull/1201

NIFI-2890 Provenance Repository Corruption

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X ] Does your PR title start with NIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [X ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X ] Is your initial contribution a single, squashed commit?

### For code changes:
- [X ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

Correct handling of corrupt journal file that was preventing instance 
startup and causing loss of records from uncorrupt files.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jskora/nifi NIFI-2890-1.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1201.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1201


commit d89d5f684e1ea19b74167a03c6b987e161fceab1
Author: Joe Skora 
Date:   2016-11-10T18:06:41Z

NIFI-2890 Provenance Repository Corruption - Correct handling of corrupt 
journal file that was preventing instance startup and causing loss of records 
from uncorrupt files.




> Provenance Repository corruption
> 
>
> Key: NIFI-2890
> URL: https://issues.apache.org/jira/browse/NIFI-2890
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Brandon DeVries
>Assignee: Joe Skora
>
> It looks as thought there was a ticket to address this\[1\], but it appears 
> to not have completely addressed the issue. On an 0.6.1 instance (that 
> crashed hard), I'm getting a stack trace on startup indicating a corrupt prov 
> repo. The relevant part of the stack trace (unfortunately typed by hand, so 
> forgive the inevitable typos) is:
> {quote}
> ...
> Caused By java.lang.IllegalArgumentException: No enum constant 
> org.apahe.nifi.provenance.ProvenenceEventType.
> at java.lang/.Enum.valueOf...
> at 
> org.apache.nifi.provenance.ProvenanaceEventType.valueOf(ProvenanaceEventType.java:19)
> at
> org.apache.nifi.provenance.StandardRecordReader.nextRecord(StandardRecordReader.java:284)
> at 
> org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1678)
>  \[2\]
> ...
> {quote}
> In 0.x HEAD, this appears to have moved a bit\[3\]. Same is true for 
> 1.x\[4\]. In both 0.x\[5\] and 1.x\[6\], code was added to check for "bad" 
> records coming from the reader, and skip them. However, this appears to only 
> address the issue if the corrupt / bad record is the *first* record coming 
> out of a reader. If it is a subsequent record, the corruption remains.
> \[1\] https://issues.apache.org/jira/browse/NIFI-803
> \[2\] 
> https://github.com/apache/nifi/blob/rel/nifi-0.6.1/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java#L1678
> \[3\] 
>

[GitHub] nifi pull request #1201: NIFI-2890 Provenance Repository Corruption

2016-11-10 Thread jskora

GitHub user jskora opened a pull request:

https://github.com/apache/nifi/pull/1201

NIFI-2890 Provenance Repository Corruption

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [X ] Does your PR title start with NIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [X ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X ] Is your initial contribution a single, squashed commit?

### For code changes:
- [X ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [X ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

Correct handling of corrupt journal file that was preventing instance 
startup and causing loss of records from uncorrupt files.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jskora/nifi NIFI-2890-1.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1201.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1201


commit d89d5f684e1ea19b74167a03c6b987e161fceab1
Author: Joe Skora 
Date:   2016-11-10T18:06:41Z

NIFI-2890 Provenance Repository Corruption - Correct handling of corrupt 
journal file that was preventing instance startup and causing loss of records 
from uncorrupt files.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines

2016-11-10 Thread Matt Burgess (JIRA)

Matt Burgess created NIFI-3021:
--

 Summary: Multi-line text boxes in View Configuration dialog (NFEL 
editors) do not honor newlines 
 Key: NIFI-3021
 URL: https://issues.apache.org/jira/browse/NIFI-3021
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Reporter: Matt Burgess


In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
pressed, a newline will be inserted into the text. This is honored for the 
Configure dialog.

However if the processor is running (such that Configure is not available and 
instead View Configuration is), the newlines are not honored in the property 
when viewed as a multi-line text box (editor/viewer).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines

2016-11-10 Thread Matt Burgess (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-3021:
---
Description: 
In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
pressed, a newline will be inserted into the text. This is honored for the 
Configure dialog.

However if the processor is running (such that Configure is not available and 
instead View Configuration is), the newlines are not honored in the property 
when viewed as a multi-line text box (editor/viewer).

One such processor that has this behavior is ExecuteScript, the Script Body 
property (see attached screenshots).

  was:
In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
pressed, a newline will be inserted into the text. This is honored for the 
Configure dialog.

However if the processor is running (such that Configure is not available and 
instead View Configuration is), the newlines are not honored in the property 
when viewed as a multi-line text box (editor/viewer).


> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).
> One such processor that has this behavior is ExecuteScript, the Script Body 
> property (see attached screenshots).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines

2016-11-10 Thread Matt Burgess (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-3021:
---
Attachment: View_Configuration_multiline.png
Configure_multiline.png

> Multi-line text boxes in View Configuration dialog (NFEL editors) do not 
> honor newlines 
> 
>
> Key: NIFI-3021
> URL: https://issues.apache.org/jira/browse/NIFI-3021
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Burgess
>  Labels: ui
> Attachments: Configure_multiline.png, View_Configuration_multiline.png
>
>
> In multi-line editors for properties in a Configure dialog, if Shift+Enter is 
> pressed, a newline will be inserted into the text. This is honored for the 
> Configure dialog.
> However if the processor is running (such that Configure is not available and 
> instead View Configuration is), the newlines are not honored in the property 
> when viewed as a multi-line text box (editor/viewer).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3020) LDAP - Support configurable user identity


 [ 
https://issues.apache.org/jira/browse/NIFI-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3020:
--
Description: 
The current LDAP provider supports a configurable search filter that will allow 
the user specified login name to be matched against any LDAP entry attribute. 
We should offer a configuration option that will indicate if we should use the 
LDAP entry DN or if we should use the login name that was used in the search 
filter. For instance, this would allow an admin to configure a user to login 
with their sAMAccountName and subsequently use that name as their user's 
identity.

Note: we should default this option to be the user DN in order to ensure 
backwards compatibility.

  was:
The current LDAP provider supports a configurable search filter that will allow 
the user specified login name to be matched against any LDAP entry attribute. 
We should offer a configuration option that will indicate if we should use the 
LDAP entry DN or if we should use the login name that was used in the search 
filter. For instance, this would allow an admin to configure a user to login 
with their sAMAccountName and subsequently use that name as their user's 
identity.

Note: we should default this option to be the user DN in order to ensure 
backwards capability.


> LDAP - Support configurable user identity
> -
>
> Key: NIFI-3020
> URL: https://issues.apache.org/jira/browse/NIFI-3020
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Gilman
>
> The current LDAP provider supports a configurable search filter that will 
> allow the user specified login name to be matched against any LDAP entry 
> attribute. We should offer a configuration option that will indicate if we 
> should use the LDAP entry DN or if we should use the login name that was used 
> in the search filter. For instance, this would allow an admin to configure a 
> user to login with their sAMAccountName and subsequently use that name as 
> their user's identity.
> Note: we should default this option to be the user DN in order to ensure 
> backwards compatibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3020) LDAP - Support configurable user identity


 [ 
https://issues.apache.org/jira/browse/NIFI-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3020:
--
Description: 
The current LDAP provider supports a configurable search filter that will allow 
the user specified login name to be matched against any LDAP entry attribute. 
We should offer a configuration option that will indicate if we should use the 
LDAP entry DN or if we should use the login name that was used in the search 
filter. For instance, this would allow an admin to configure a user to login 
with their sAMAccountName and subsequently use that name as their user's 
identity.

Note: we should default this option to be the user DN in order to ensure 
backwards capability.

  was:
The current LDAP provider supports a configurable search filter that will allow 
the user specified login name to be matched against any LDAP entry attribute. 
We should offer a configuration option that will indicate if we should use the 
LDAP entry DN or if we should use the login name that was used in the search 
filter. This would allow an admin to configure a user to login with their 
sAMAccountName and subsequently use that name as their user's identity.

Note: we should default this option to be the user DN in order to ensure 
backwards capability.


> LDAP - Support configurable user identity
> -
>
> Key: NIFI-3020
> URL: https://issues.apache.org/jira/browse/NIFI-3020
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Gilman
>
> The current LDAP provider supports a configurable search filter that will 
> allow the user specified login name to be matched against any LDAP entry 
> attribute. We should offer a configuration option that will indicate if we 
> should use the LDAP entry DN or if we should use the login name that was used 
> in the search filter. For instance, this would allow an admin to configure a 
> user to login with their sAMAccountName and subsequently use that name as 
> their user's identity.
> Note: we should default this option to be the user DN in order to ensure 
> backwards capability.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3020) LDAP - Support configurable user identity

Matt Gilman created NIFI-3020:
-

 Summary: LDAP - Support configurable user identity
 Key: NIFI-3020
 URL: https://issues.apache.org/jira/browse/NIFI-3020
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Extensions
Reporter: Matt Gilman


The current LDAP provider supports a configurable search filter that will allow 
the user specified login name to be matched against any LDAP entry attribute. 
We should offer a configuration option that will indicate if we should use the 
LDAP entry DN or if we should use the login name that was used in the search 
filter. This would allow an admin to configure a user to login with their 
sAMAccountName and subsequently use that name as their user's identity.

Note: we should default this option to be the user DN in order to ensure 
backwards capability.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1193: NIFI-2957 ZooKeeper Migration Toolkit

2016-11-10 Thread jtstorck

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@brosander Code has been updated to allow sending data to the root of a 
Zookeeper server.  Also, you shouldn't have to specify the port when sending 
data if Zookeeper is running on the default port.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit

2016-11-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654616#comment-15654616
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@brosander Code has been updated to allow sending data to the root of a 
Zookeeper server.  Also, you shouldn't have to specify the port when sending 
data if Zookeeper is running on the default port.


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a destination ZooKeeper
> * The toolkit will not allow data to be written to the same ZooKeeper from 
> which the source data was obtained.
> * The toolkit will not support reconnecting to ZooKeeper if it is 
> disconnected.  The user can rerun the tool.
> * The toolkit will support ZooKeepers configured with Kerberos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3009) UI - NaN error when backpressure not configured


[ 
https://issues.apache.org/jira/browse/NIFI-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654548#comment-15654548
 ] 

ASF subversion and git services commented on NIFI-3009:
---

Commit f83863ebae700ade07aa8b93ac1e2d82a6cf053c in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f83863e ]

NIFI-3009:
- Fixing NaN error when backpressure is not configured.
This closes #1200


> UI - NaN error when backpressure not configured
> ---
>
> Key: NIFI-3009
> URL: https://issues.apache.org/jira/browse/NIFI-3009
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> When backpressure is not configured on a queue, the console shows the 
> following errors:
> {code}
> Error:  attribute width: Expected length, "NaN".
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3009) UI - NaN error when backpressure not configured

2016-11-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654550#comment-15654550
 ] 

ASF GitHub Bot commented on NIFI-3009:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1200


> UI - NaN error when backpressure not configured
> ---
>
> Key: NIFI-3009
> URL: https://issues.apache.org/jira/browse/NIFI-3009
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> When backpressure is not configured on a queue, the console shows the 
> following errors:
> {code}
> Error:  attribute width: Expected length, "NaN".
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1200: UI - Address NaN error when backpressure is not con...

2016-11-10 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1200


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3009) UI - NaN error when backpressure not configured


[ 
https://issues.apache.org/jira/browse/NIFI-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654502#comment-15654502
 ] 

ASF subversion and git services commented on NIFI-3009:
---

Commit f83863ebae700ade07aa8b93ac1e2d82a6cf053c in nifi's branch 
refs/heads/apache-master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f83863e ]

NIFI-3009:
- Fixing NaN error when backpressure is not configured.
This closes #1200


> UI - NaN error when backpressure not configured
> ---
>
> Key: NIFI-3009
> URL: https://issues.apache.org/jira/browse/NIFI-3009
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> When backpressure is not configured on a queue, the console shows the 
> following errors:
> {code}
> Error:  attribute width: Expected length, "NaN".
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654447#comment-15654447
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@jtstorck I'm having some problems trying to save a path (/node) and then 
restore at the same path in a new zookeeper instance.

Here's what I'm getting (with a teardown and reinstantiation of the 
zookeeper container between receive and send commands) :
```
zk-migrator.sh -z zoo1/node -r -f /out/out.json
zk-migrator.sh -z zoo1:2181/ -s -f /out/out.json
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
SLF4J: Actual binding is of type 
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2016-11-10 16:14:26,590 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 
Source data was obtained from ZooKeeper: 
ZooKeeperEndpointConfig{connectString=zoo1, path=/node}
2016-11-10 16:14:26,621 INFO [ForkJoinPool.commonPool-worker-1] 
o.a.n.t.zkmigrator.ZooKeeperMigrator transformed original node 
DataStatAclNode{path=/node, acls=[31,s{'world,'anyone}
], ephemeralOwner=0} to DataStatAclNode{path=//node, 
acls=[31,s{'world,'anyone}
], ephemeralOwner=0}
Exception in thread "main" java.lang.RuntimeException: unable to perform 
operation: java.lang.IllegalArgumentException: Invalid path string "//node" 
caused by empty node name specified @1
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:145)
Caused by: java.util.concurrent.ExecutionException: 
java.lang.IllegalArgumentException: Invalid path string "//node" caused by 
empty node name specified @1
at 
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at 
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.writeZooKeeper(ZooKeeperMigrator.java:171)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:139)
Caused by: java.lang.IllegalArgumentException: Invalid path string "//node" 
caused by empty node name specified @1
at 
org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:99)
at 
org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:766)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.ensureNodeExists(ZooKeeperMigrator.java:214)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.lambda$null$5(ZooKeeperMigrator.java:159)
at 
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at 
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at 
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at 
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
at 
java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at 
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at 
java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at 
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
```


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a

[GitHub] nifi issue #1193: NIFI-2957 ZooKeeper Migration Toolkit

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@jtstorck I'm having some problems trying to save a path (/node) and then 
restore at the same path in a new zookeeper instance.

Here's what I'm getting (with a teardown and reinstantiation of the 
zookeeper container between receive and send commands) :
```
zk-migrator.sh -z zoo1/node -r -f /out/out.json
zk-migrator.sh -z zoo1:2181/ -s -f /out/out.json
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
SLF4J: Actual binding is of type 
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2016-11-10 16:14:26,590 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 
Source data was obtained from ZooKeeper: 
ZooKeeperEndpointConfig{connectString=zoo1, path=/node}
2016-11-10 16:14:26,621 INFO [ForkJoinPool.commonPool-worker-1] 
o.a.n.t.zkmigrator.ZooKeeperMigrator transformed original node 
DataStatAclNode{path=/node, acls=[31,s{'world,'anyone}
], ephemeralOwner=0} to DataStatAclNode{path=//node, 
acls=[31,s{'world,'anyone}
], ephemeralOwner=0}
Exception in thread "main" java.lang.RuntimeException: unable to perform 
operation: java.lang.IllegalArgumentException: Invalid path string "//node" 
caused by empty node name specified @1
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:145)
Caused by: java.util.concurrent.ExecutionException: 
java.lang.IllegalArgumentException: Invalid path string "//node" caused by 
empty node name specified @1
at 
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at 
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.writeZooKeeper(ZooKeeperMigrator.java:171)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorMain.main(ZooKeeperMigratorMain.java:139)
Caused by: java.lang.IllegalArgumentException: Invalid path string "//node" 
caused by empty node name specified @1
at 
org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:99)
at 
org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:766)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.ensureNodeExists(ZooKeeperMigrator.java:214)
at 
org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigrator.lambda$null$5(ZooKeeperMigrator.java:159)
at 
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at 
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at 
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at 
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
at 
java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at 
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at 
java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at 
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit


[ 
https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654390#comment-15654390
 ] 

ASF GitHub Bot commented on NIFI-2957:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@jtstorck I'm getting logging messages in stdout when I don't specify a 
file:

```
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
SLF4J: Actual binding is of type 
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2016-11-10 15:53:36,446 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 
Persisting data from source ZooKeeper: 
ZooKeeperEndpointConfig{connectString=zoo1, path=/node}
[
  {
"connectString": "zoo1",
"path": "/node"
  },
  {
"path": "/node",
"data": [
  99,
  111,
  110,
  116,
  101,
  110,
  116
],
"stat": {
  "czxid": 9,
  "mzxid": 9,
  "ctime": 1478792367151,
  "mtime": 1478792367151,
  "version": 0,
  "cversion": 0,
  "aversion": 0,
  "ephemeralOwner": 0,
  "dataLength": 7,
  "numChildren": 0,
  "pzxid": 9
},
"acls": [
  {
"perms": 31,
"id": {
  "scheme": "world",
  "id": "anyone"
}
  }
],
"ephemeralOwner": 0
  }
]2016-11-10 15:53:36,496 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 1 
nodes read from ZooKeeperEndpointConfig{connectString=zoo1, path=/node}
```


> ZooKeeper migration toolkit
> ---
>
> Key: NIFI-2957
> URL: https://issues.apache.org/jira/browse/NIFI-2957
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Tools and Build
>Affects Versions: 1.0.0, 0.7.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
> Fix For: 1.2.0
>
>
> When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the 
> embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be 
> migrated.
> Initial considerations:
> * Username/password protection of nodes is not supported in NiFi 1.x.. Those 
> nodes that are configured that way in ZooKeeper need to be migrated to have 
> an open ACL.
> * The toolkit will support a mode to read data from a configurable root node 
> in a source ZooKeeper, and the data will be written to a file designated via 
> CLI.
> * The toolkit will support a mode to write data to a destination ZooKeeper
> * The toolkit will not allow data to be written to the same ZooKeeper from 
> which the source data was obtained.
> * The toolkit will not support reconnecting to ZooKeeper if it is 
> disconnected.  The user can rerun the tool.
> * The toolkit will support ZooKeepers configured with Kerberos.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1193: NIFI-2957 ZooKeeper Migration Toolkit

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/1193
  
@jtstorck I'm getting logging messages in stdout when I don't specify a 
file:

```
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/nifi-toolkit/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
SLF4J: Actual binding is of type 
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2016-11-10 15:53:36,446 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 
Persisting data from source ZooKeeper: 
ZooKeeperEndpointConfig{connectString=zoo1, path=/node}
[
  {
"connectString": "zoo1",
"path": "/node"
  },
  {
"path": "/node",
"data": [
  99,
  111,
  110,
  116,
  101,
  110,
  116
],
"stat": {
  "czxid": 9,
  "mzxid": 9,
  "ctime": 1478792367151,
  "mtime": 1478792367151,
  "version": 0,
  "cversion": 0,
  "aversion": 0,
  "ephemeralOwner": 0,
  "dataLength": 7,
  "numChildren": 0,
  "pzxid": 9
},
"acls": [
  {
"perms": 31,
"id": {
  "scheme": "world",
  "id": "anyone"
}
  }
],
"ephemeralOwner": 0
  }
]2016-11-10 15:53:36,496 INFO [main] o.a.n.t.zkmigrator.ZooKeeperMigrator 1 
nodes read from ZooKeeperEndpointConfig{connectString=zoo1, path=/node}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1200: UI - Address NaN error when backpressure is not configured