[jira] [Updated] (NIFI-3363) PutKafka throws NullPointerException when User-Defined partition strategy is used

[Koji Kawamura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Kawamura updated NIFI-3363:

Summary: PutKafka throws NullPointerException when User-Defined partition 
strategy is used  (was: Putkafka processor throws null pointer exception when 
we choose User-Defined strategy)

> PutKafka throws NullPointerException when User-Defined partition strategy is 
> used
> -
>
> Key: NIFI-3363
> URL: https://issues.apache.org/jira/browse/NIFI-3363
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0, 0.4.0, 0.5.0, 0.6.0, 0.4.1, 0.5.1, 0.7.0, 0.6.1, 
> 1.1.0, 0.7.1, 1.1.1, 1.0.1
> Environment: By looking at the release tags contained in a commit 
> that added User-Defined partition strategy (NIFI-1097 22de23b), it seems all 
> NiFi versions since 0.4.0 is affected.
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> NullPointerException is thrown because PutKafka tries to put null into 
> properties since following if statements doesn't cover 
> USER_DEFINED_PARTITIONING.
> {code: title=PutKafka.java buildKafkaConfigProperties}
> String partitionStrategy = context.getProperty(PARTITION_STRATEGY).getValue();
> String partitionerClass = null;
> if (partitionStrategy.equalsIgnoreCase(ROUND_ROBIN_PARTITIONING.getValue())) {
> partitionerClass = Partitioners.RoundRobinPartitioner.class.getName();
> } else if 
> (partitionStrategy.equalsIgnoreCase(RANDOM_PARTITIONING.getValue())) {
> partitionerClass = Partitioners.RandomPartitioner.class.getName();
> }
> properties.setProperty("partitioner.class", partitionerClass); // Happens here
> {code}
> A naive fix for this would be adding one more if statement so that it puts 
> 'partitioner.class' property only if partitionerClass is set.
> However, while I was testing the fix, I found following facts, that revealed 
> this approach wouldn't be the right solution for this issue.
> In short, we don't have to set 'partitioner.class' property with Kafka 0.8.x 
> client in the first place. I assume it's there because PutKafka came through 
> a long history..
> h2. PutKafka history analysis
> - PutKafka used to cover Kafka 0.8 and 0.9
> - Around the time Kafka 0.9 was released, PutKafka added 'partitioner.class' 
> via NIFI-1097: start using new API. There were two client libraries 
> kafka-clients and kafka_2.9.1, both 0.8.2.2.
> - Then PublishKafka is added for Kafka 0.9, at this point, we could add 
> 'partition' property to PublishKafka, but we didn't do that for some reason. 
> PublishKafka doesn't support user defined partition as of this writing. 
> NIFI-1296.
> - The code adding 'partitioner.class' has been left in PutKafka.
> - Further, we separated nar into 0.8, 0.9 and 0.10.
> - Now only PutKafka(0.8) uses 'partitioner.class' property, but 0.8 client 
> doesn't use that property. So we don't need that code at all.
> h2. Then, how should we fix this?
> Since PutKafka in both master and 0.x branch specifically uses Kafka 0.8.x 
> client. We can simply remove the codes adding 'partitioner.class', probably 
> PARTITION_STRATEGY processor property, too.
> h2. Expected result after fix
> - Users can specify Kafka partition with PutKafka 'partition' property, but 
> no need to specify 'partition strategy', NullPointerException won't be thrown
> - A warning log, that used to be logged in nifi-app.log won't be logged any 
> more: {code}2017-01-18 13:53:33,071 WARN [Timer-Driven Process Thread-9] 
> o.a.k.clients.producer.ProducerConfig The configuration partitioner.class = 
> null was supplied but isn't a known config.{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3363) Putkafka processor throws null pointer exception when we choose User-Defined strategy

[Koji Kawamura (JIRA)]

Koji Kawamura created NIFI-3363:
---

 Summary: Putkafka processor throws null pointer exception when we 
choose User-Defined strategy
 Key: NIFI-3363
 URL: https://issues.apache.org/jira/browse/NIFI-3363
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Affects Versions: 1.0.1, 1.1.1, 0.7.1, 1.1.0, 0.6.1, 0.7.0, 0.5.1, 0.4.1, 
0.6.0, 0.5.0, 0.4.0, 1.0.0
 Environment: By looking at the release tags contained in a commit that 
added User-Defined partition strategy (NIFI-1097 22de23b), it seems all NiFi 
versions since 0.4.0 is affected.
Reporter: Koji Kawamura
Assignee: Koji Kawamura


NullPointerException is thrown because PutKafka tries to put null into 
properties since following if statements doesn't cover 
USER_DEFINED_PARTITIONING.

{code: title=PutKafka.java buildKafkaConfigProperties}
String partitionStrategy = context.getProperty(PARTITION_STRATEGY).getValue();
String partitionerClass = null;
if (partitionStrategy.equalsIgnoreCase(ROUND_ROBIN_PARTITIONING.getValue())) {
partitionerClass = Partitioners.RoundRobinPartitioner.class.getName();
} else if (partitionStrategy.equalsIgnoreCase(RANDOM_PARTITIONING.getValue())) {
partitionerClass = Partitioners.RandomPartitioner.class.getName();
}
properties.setProperty("partitioner.class", partitionerClass); // Happens here
{code}

A naive fix for this would be adding one more if statement so that it puts 
'partitioner.class' property only if partitionerClass is set.

However, while I was testing the fix, I found following facts, that revealed 
this approach wouldn't be the right solution for this issue.

In short, we don't have to set 'partitioner.class' property with Kafka 0.8.x 
client in the first place. I assume it's there because PutKafka came through a 
long history..

h2. PutKafka history analysis

- PutKafka used to cover Kafka 0.8 and 0.9
- Around the time Kafka 0.9 was released, PutKafka added 'partitioner.class' 
via NIFI-1097: start using new API. There were two client libraries 
kafka-clients and kafka_2.9.1, both 0.8.2.2.
- Then PublishKafka is added for Kafka 0.9, at this point, we could add 
'partition' property to PublishKafka, but we didn't do that for some reason. 
PublishKafka doesn't support user defined partition as of this writing. 
NIFI-1296.
- The code adding 'partitioner.class' has been left in PutKafka.
- Further, we separated nar into 0.8, 0.9 and 0.10.
- Now only PutKafka(0.8) uses 'partitioner.class' property, but 0.8 client 
doesn't use that property. So we don't need that code at all.

h2. Then, how should we fix this?

Since PutKafka in both master and 0.x branch specifically uses Kafka 0.8.x 
client. We can simply remove the codes adding 'partitioner.class', probably 
PARTITION_STRATEGY processor property, too.

h2. Expected result after fix

- Users can specify Kafka partition with PutKafka 'partition' property, but no 
need to specify 'partition strategy', NullPointerException won't be thrown
- A warning log, that used to be logged in nifi-app.log won't be logged any 
more: {code}2017-01-18 13:53:33,071 WARN [Timer-Driven Process Thread-9] 
o.a.k.clients.producer.ProducerConfig The configuration partitioner.class = 
null was supplied but isn't a known config.{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (NIFI-3291) UI - Upgrade jQuery

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reopened NIFI-3291:
---

Looks like there are some style issues with the auto complete widgets after 
this upgrade. Please address.

> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3362) Update FlowConfiguration.xsd TimePeriod to match FormatUtils

[Michael Moser (JIRA)]

Michael Moser created NIFI-3362:
---

 Summary: Update FlowConfiguration.xsd TimePeriod to match 
FormatUtils
 Key: NIFI-3362
 URL: https://issues.apache.org/jira/browse/NIFI-3362
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.1.1, 0.7.1
Reporter: Michael Moser
Priority: Minor


The framework FormatUtils time duration parser has been updated to support new 
terms such as "weeks".  But when using those new terms, we get ERROR messages 
in the nifi-bootstrap.log.

cvc-pattern-valid: Value '4 weeks' is not facet-valid with respect to pattern 
'\d+\s*(ns|nano|nanos|nanoseconds|ms|milli|millis|milliseconds|s|sec|secs|seconds|m|min|mins|minutes|h|hr|hrs|hours|d|day|days)'
 for type 'TimePeriod'.
cvc-type.3.1.3: The value '4 weeks' of element 'flowFileExpiration' is not 
valid.

FlowConfiguration.xsd TimePeriod pattern should be updated to match FormatUtils.

Add to FlowConguration.xsd:
nanosecond
millisecond
second
minute
hour
w
wk
wks
week
weeks

Add to FormatUtils.java
nanosecond
millisecond



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3361) Upgrade Jetty

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-3361:
-

 Summary: Upgrade Jetty
 Key: NIFI-3361
 URL: https://issues.apache.org/jira/browse/NIFI-3361
 Project: Apache NiFi
  Issue Type: Task
  Components: Core Framework
Reporter: Matt Gilman
Assignee: Matt Gilman
 Fix For: 1.2.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1423: Merge pull request

[apiri]

Github user apiri commented on the issue:

https://github.com/apache/nifi/pull/1423
  
Hey @zmyer,

Were you intending to submit a PR or was this created on accident?  If you 
were trying to submit a contribution, it looks like something went awry.  
Please let us know if you would like some assistance.  In the case of the 
latter, could you please close this item?

Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3291) UI - Upgrade jQuery

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826794#comment-15826794
 ] 

ASF GitHub Bot commented on NIFI-3291:
--

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/1408
  
Thanks @mcgilman this has been merged to master.


> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1408: NIFI-3291: Upgrade jQuery and plugins

[scottyaslan]

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/1408
  
Thanks @mcgilman this has been merged to master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3291) UI - Upgrade jQuery

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-3291:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3291) UI - Upgrade jQuery

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826772#comment-15826772
 ] 

ASF GitHub Bot commented on NIFI-3291:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1408


> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3291) UI - Upgrade jQuery

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826769#comment-15826769
 ] 

ASF subversion and git services commented on NIFI-3291:
---

Commit 82cf0c6fa87ff8e5057b94020ca935daa4381289 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=82cf0c6 ]

NIFI-3291:
- Ensuring we don't see a horizontal scrollbar.
- Ensuring slick-nifi-theme is bundled.

This closes #1408


> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1408: NIFI-3291: Upgrade jQuery and plugins

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1408


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3291) UI - Upgrade jQuery

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826768#comment-15826768
 ] 

ASF subversion and git services commented on NIFI-3291:
---

Commit 6170f6442ed45dd084617cb4f39085640f49ebc5 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6170f64 ]

NIFI-3291:
- Upgrading jQuery and it's plugins.
- Packaging at build time using npm.


> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1408: NIFI-3291: Upgrade jQuery and plugins

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1408
  
Thanks. Just updated the PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3291) UI - Upgrade jQuery

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826691#comment-15826691
 ] 

ASF GitHub Bot commented on NIFI-3291:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1408
  
Thanks. Just updated the PR.


> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3360) Modularize canvas directory

[Scott Aslan (JIRA)]

Scott Aslan created NIFI-3360:
-

 Summary: Modularize canvas directory
 Key: NIFI-3360
 URL: https://issues.apache.org/jira/browse/NIFI-3360
 Project: Apache NiFi
  Issue Type: Sub-task
  Components: Core UI
Reporter: Scott Aslan
Assignee: Scott Aslan


Need to modularize (re-package) UI components to promote maintainability and 
eventual reusability for UI extensions. Components are already conceptually 
modular however circular references exists which should not be allowed. Need to 
re-package functions to eliminate circular
references.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3359) Modularize all of nifi-web-ui except canvas directory

[Scott Aslan (JIRA)]

Scott Aslan created NIFI-3359:
-

 Summary: Modularize all of nifi-web-ui except canvas directory
 Key: NIFI-3359
 URL: https://issues.apache.org/jira/browse/NIFI-3359
 Project: Apache NiFi
  Issue Type: Sub-task
  Components: Core UI
Reporter: Scott Aslan
Assignee: Scott Aslan


Need to modularize (re-package) UI components to promote maintainability and 
eventual reusability for UI extensions. Components are already conceptually 
modular however circular references exists which should not be allowed. Need to 
re-package functions to eliminate circular
references.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3358) UI Maintenance

[Scott Aslan (JIRA)]

Scott Aslan created NIFI-3358:
-

 Summary: UI Maintenance
 Key: NIFI-3358
 URL: https://issues.apache.org/jira/browse/NIFI-3358
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Scott Aslan


This is an initial parent ticket to capture work and efforts to enhance the UI 
components to promote maintainability and eventual reusability for UI 
extensions.

Subcomponents of this effort should be treated as subtasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-3340:
--
Fix Version/s: 1.2.0

> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
> Fix For: 1.2.0
>
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> The directory will be created if it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende resolved NIFI-3340.
---
Resolution: Fixed

> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
> Fix For: 1.2.0
>
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> The directory will be created if it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826561#comment-15826561
 ] 

ASF GitHub Bot commented on NIFI-3340:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1424


> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> The directory will be created if it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1424: NIFI-3340 Modify description of the Directory prope...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1424


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826560#comment-15826560
 ] 

ASF subversion and git services commented on NIFI-3340:
---

Commit ad98ac179f69bc938ac4c972548a6fa205074fe7 in nifi's branch 
refs/heads/master from [~andrewmlim]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ad98ac1 ]

NIFI-3340 Modify description of the Directory property for PutHDFS to say it 
will create the directory if it doesn't exist

This closes #1424.

Signed-off-by: Bryan Bende 


> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> The directory will be created if it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826554#comment-15826554
 ] 

ASF GitHub Bot commented on NIFI-3340:
--

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/1424
  
+1 Will merge


> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> The directory will be created if it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1424: NIFI-3340 Modify description of the Directory property for...

[bbende]

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/1424
  
+1 Will merge


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3300) Zookeeper Migrator should allow importing of data to a new root node

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-3300:
--
Description: 
ZooKeeper Migrator exports data from ZooKeeper using the absolute path from the 
root of ZooKeeper.  This prevents the importing of data to a new root node for 
NiFi, since the path given during the import will have the entire path of the 
exported data appended to the new root.

For example, if "/nifi/components" is exported from a ZooKeeper server, the 
exported data will include the "/nifi/components" path.  When that data is 
imported to a different ZooKeeper server where the root NiFi node is "/nifi2", 
and the user imports that data to "/nifi2", nodes will be created under 
"/nifi2/nifi/components".

The ZooKeeper Migrator should export data in such a way that, with the given 
example, the source nodes under "/nifi/components" should be exported without 
the "/nifi/components" portion of the path, so that those nodes could be 
imported to the destination root path, such as "/nifi2/components".

Usage of ZooKeeper client's "chroot" capability should be used in favor of the 
custom pathing code in the ZooKeeper Migrator.

This will require documentation updates in the ZooKeeper Migrator section of 
the System Administration Guide.

  was:
ZooKeeper Migrator exports data from ZooKeeper using the absolute path from the 
root of ZooKeeper.  This prevents the importing of data to a new root node for 
NiFi, since the path given during the import will have the entire path of the 
exported data appended to the new root.

For example, if "/nifi/components" is exported from a ZooKeeper server, the 
exported data will include the "/nifi/components" path.  When that data is 
imported to a different ZooKeeper server where the root NiFi node is "/nifi2", 
and the user imports that data to "/nifi2", nodes will be created under 
"/nifi2/nifi/components".

The ZooKeeper Migrator should export data in such a way that, with the given 
example, the source nodes under "/nifi/components" should be exported without 
the "/nifi/components" portion of the path, so that those nodes could be 
imported to the destination root path, such as "/nifi2/components".

Usage of ZooKeeper client's "chroot" capability should be used in favor of the 
custom pathing code in the ZooKeeper Migrator.


> Zookeeper Migrator should allow importing of data to a new root node
> 
>
> Key: NIFI-3300
> URL: https://issues.apache.org/jira/browse/NIFI-3300
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Minor
>
> ZooKeeper Migrator exports data from ZooKeeper using the absolute path from 
> the root of ZooKeeper.  This prevents the importing of data to a new root 
> node for NiFi, since the path given during the import will have the entire 
> path of the exported data appended to the new root.
> For example, if "/nifi/components" is exported from a ZooKeeper server, the 
> exported data will include the "/nifi/components" path.  When that data is 
> imported to a different ZooKeeper server where the root NiFi node is 
> "/nifi2", and the user imports that data to "/nifi2", nodes will be created 
> under "/nifi2/nifi/components".
> The ZooKeeper Migrator should export data in such a way that, with the given 
> example, the source nodes under "/nifi/components" should be exported without 
> the "/nifi/components" portion of the path, so that those nodes could be 
> imported to the destination root path, such as "/nifi2/components".
> Usage of ZooKeeper client's "chroot" capability should be used in favor of 
> the custom pathing code in the ZooKeeper Migrator.
> This will require documentation updates in the ZooKeeper Migrator section of 
> the System Administration Guide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3158) ZooKeeper Migrator should be better about cleaning up resources and throw more specific exceptions

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-3158:
--
Fix Version/s: (was: 1.2.0)

> ZooKeeper Migrator should be better about cleaning up resources and throw 
> more specific exceptions
> --
>
> Key: NIFI-3158
> URL: https://issues.apache.org/jira/browse/NIFI-3158
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Minor
>
> ZooKeeper Migrator should close input/output streams and the ZK client when 
> it's finished using them.  Also, IOExceptions should be used in most places 
> that currently throw RuntimeException.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3330) HandleHttpRequest intermittently throwing NullPointerException

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-3330:
--
Fix Version/s: 1.2.0

> HandleHttpRequest intermittently throwing NullPointerException
> --
>
> Key: NIFI-3330
> URL: https://issues.apache.org/jira/browse/NIFI-3330
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Minor
> Fix For: 1.2.0
>
>
> HandleHttpRequest intermittently throws a NullPointerException, specifically 
> seen while starting up a node that is part of a cluster in which the 
> processors are in a started state.  The exception is occurring on a line 
> which appears to have a null check before it, but it's a null check on the 
> result of a method, not a variable.
> {code}2017-01-11 14:34:13,708 ERROR [Timer-Driven Process Thread-220] 
> o.a.n.p.standard.HandleHttpRequest 
> HandleHttpRequest[id=e00f0d3b-0158-1000-1e3f-81d710d3d7c9] 
> HandleHttpRequest[id=e00f0d3b-0158-1000-1e3f-81d710d3d7c9] failed to process 
> due to j
> ava.lang.NullPointerException; rolling back session: 
> java.lang.NullPointerException
> 2017-01-11 14:34:13,996 ERROR [Timer-Driven Process Thread-220] 
> o.a.n.p.standard.HandleHttpRequest
> java.lang.NullPointerException: null
> at 
> org.apache.nifi.processors.standard.HandleHttpRequest.onTrigger(HandleHttpRequest.java:527)
>  ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  ~[nifi-api-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_111]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_111]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_111]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_111]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]{code}
> The following section (and other sections similar to it) in the onTrigger 
> method:
> {code}if (request.getDispatcherType() != null) {
> putAttribute(attributes, "http.dispatcher.type", 
> request.getDispatcherType().name());
> }{code}
> should be updated to save the result of the method and then check for null 
> before operating on that value.
> For example:
> {code}String dispatcherType = request.getDispatcherType();
> if (dispatcherType != null) {
> putAttribute(attributes, "http.dispatcher.type", 
> dispatcherType.name());
> }{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-3330) HandleHttpRequest intermittently throwing NullPointerException

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck resolved NIFI-3330.
---
Resolution: Fixed

> HandleHttpRequest intermittently throwing NullPointerException
> --
>
> Key: NIFI-3330
> URL: https://issues.apache.org/jira/browse/NIFI-3330
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.1.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Minor
> Fix For: 1.2.0
>
>
> HandleHttpRequest intermittently throws a NullPointerException, specifically 
> seen while starting up a node that is part of a cluster in which the 
> processors are in a started state.  The exception is occurring on a line 
> which appears to have a null check before it, but it's a null check on the 
> result of a method, not a variable.
> {code}2017-01-11 14:34:13,708 ERROR [Timer-Driven Process Thread-220] 
> o.a.n.p.standard.HandleHttpRequest 
> HandleHttpRequest[id=e00f0d3b-0158-1000-1e3f-81d710d3d7c9] 
> HandleHttpRequest[id=e00f0d3b-0158-1000-1e3f-81d710d3d7c9] failed to process 
> due to j
> ava.lang.NullPointerException; rolling back session: 
> java.lang.NullPointerException
> 2017-01-11 14:34:13,996 ERROR [Timer-Driven Process Thread-220] 
> o.a.n.p.standard.HandleHttpRequest
> java.lang.NullPointerException: null
> at 
> org.apache.nifi.processors.standard.HandleHttpRequest.onTrigger(HandleHttpRequest.java:527)
>  ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  ~[nifi-api-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_111]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_111]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_111]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_111]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]{code}
> The following section (and other sections similar to it) in the onTrigger 
> method:
> {code}if (request.getDispatcherType() != null) {
> putAttribute(attributes, "http.dispatcher.type", 
> request.getDispatcherType().name());
> }{code}
> should be updated to save the result of the method and then check for null 
> before operating on that value.
> For example:
> {code}String dispatcherType = request.getDispatcherType();
> if (dispatcherType != null) {
> putAttribute(attributes, "http.dispatcher.type", 
> dispatcherType.name());
> }{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2881) Allow Database Fetch processor(s) to accept incoming flow files and use Expression Language

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826349#comment-15826349
 ] 

ASF GitHub Bot commented on NIFI-2881:
--

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1407
  
If we going to add the ability to keep track of dynamic max-value 
properties in GenerateTableFetch, then there's really no reason why we can't do 
the same for QueryDatabaseTable. I left it out of the original Jira based on 
the premise that we wouldn't support max-value columns with incoming 
connections, and at that point QueryDatabaseTable would work basically like 
ExecuteSQL.
The major difference in this case will be that if there is an incoming 
connection, then the fetch processors are purely event-driven, meaning they 
will need to be hydrated from upstream to get new results. If there is no 
incoming connection, then they will maintain their current behavior of running 
on their current schedule to check for new max values.  Does this sound ok?


> Allow Database Fetch processor(s) to accept incoming flow files and use 
> Expression Language
> ---
>
> Key: NIFI-2881
> URL: https://issues.apache.org/jira/browse/NIFI-2881
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>
> The QueryDatabaseTable and GenerateTableFetch processors do not allow 
> Expression Language to be used in the properties, mainly because they also do 
> not allow incoming connections. This means if the user desires to fetch from 
> multiple tables, they currently need one instance of the processor for each 
> table, and those table names must be hard-coded.
> To support the same capabilities for multiple tables and more flexible 
> configuration via Expression Language, these processors should have 
> properties that accept Expression Language, and GenerateTableFetch should 
> accept (optional) incoming connections.
> Conversation about the behavior of the processors is welcomed and encouraged. 
> For example, if an incoming flow file is available, do we also still run the 
> incremental fetch logic for tables that aren't specified by this flow file, 
> or do we just do incremental fetching when the processor is scheduled but 
> there is no incoming flow file. The latter implies a denial-of-service could 
> take place, by flooding the processor with flow files and not letting it do 
> its original job of querying the table, keeping track of maximum values, etc.
> This is likely a breaking change to the processors because of how state 
> management is implemented. Currently since the table name is hard coded, only 
> the column name comprises the key in the state. This would have to be 
> extended to have a compound key that represents table name, max-value column 
> name, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1407: NIFI-2881: Added EL support to DB Fetch processors, allow ...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1407
  
If we going to add the ability to keep track of dynamic max-value 
properties in GenerateTableFetch, then there's really no reason why we can't do 
the same for QueryDatabaseTable. I left it out of the original Jira based on 
the premise that we wouldn't support max-value columns with incoming 
connections, and at that point QueryDatabaseTable would work basically like 
ExecuteSQL.
The major difference in this case will be that if there is an incoming 
connection, then the fetch processors are purely event-driven, meaning they 
will need to be hydrated from upstream to get new results. If there is no 
incoming connection, then they will maintain their current behavior of running 
on their current schedule to check for new max values.  Does this sound ok?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826347#comment-15826347
 ] 

ASF GitHub Bot commented on NIFI-3340:
--

GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi/pull/1424

NIFI-3340 Modify description of the Directory property for PutHDFS to…

… say it will create the directory if it doesn't exist

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi NIFI-3340

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1424






> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> The directory will be created if it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1424: NIFI-3340 Modify description of the Directory prope...

[andrewmlim]

GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi/pull/1424

NIFI-3340 Modify description of the Directory property for PutHDFS to…

… say it will create the directory if it doesn't exist

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi NIFI-3340

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1424






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[Andrew Lim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lim updated NIFI-3340:
-
Description: 
The current description for the "Directory" property for PutHDFS only states:

The parent HDFS directory to which files should be written

It would be helpful to add:

The directory will be created if it doesn't exist.

  was:
The current description for the "Directory" property for PutHDFS only states:

The parent HDFS directory to which files should be written

It would be helpful to add:

Missing destination directories will be created.


> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> The directory will be created if it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3340) Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist

[Andrew Lim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lim updated NIFI-3340:
-
Description: 
The current description for the "Directory" property for PutHDFS only states:

The parent HDFS directory to which files should be written

It would be helpful to add:

Missing destination directories will be created.

  was:
The current description for the "Directory" property for PutHDFS only states:

The parent HDFS directory to which files should be written
Supports Expression Language: true

It would be helpful to add:

Missing destination directories will be created.


> Modify description of the Directory property for PutHDFS to say it will 
> create the directory if it doesn't exist
> 
>
> Key: NIFI-3340
> URL: https://issues.apache.org/jira/browse/NIFI-3340
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
>
> The current description for the "Directory" property for PutHDFS only states:
> The parent HDFS directory to which files should be written
> It would be helpful to add:
> Missing destination directories will be created.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3291) UI - Upgrade jQuery

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826303#comment-15826303
 ] 

ASF GitHub Bot commented on NIFI-3291:
--

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/1408
  
I also noticed there are scrollbars showing up in the "Access Policies" 
shell as well as the "Add Reporting Task" dialog.

![policies scrollbar 
issue](https://cloud.githubusercontent.com/assets/6797571/22028151/4406ac68-dca4-11e6-8260-25af3508fc25.png)

![rt scrollbar 
issue](https://cloud.githubusercontent.com/assets/6797571/22028163/4e128664-dca4-11e6-9963-5feb45c2e1fb.png)



> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1408: NIFI-3291: Upgrade jQuery and plugins

[scottyaslan]

Github user scottyaslan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1408#discussion_r96442571
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
 ---
@@ -710,8 +784,6 @@
 
 
 assets/**/*,
-css/reset.css,
-css/reset.css.gz,
 css/common-ui.css,
 css/common-ui.css.gz,
 css/message-page.css,
--- End diff --

We need to add the slick-nifi-theme here:

css/slick-nifi-theme.css,
css/slick-nifi-theme.css.gz,


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1408: NIFI-3291: Upgrade jQuery and plugins

[scottyaslan]

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/1408
  
I also noticed there are scrollbars showing up in the "Access Policies" 
shell as well as the "Add Reporting Task" dialog.

![policies scrollbar 
issue](https://cloud.githubusercontent.com/assets/6797571/22028151/4406ac68-dca4-11e6-8260-25af3508fc25.png)

![rt scrollbar 
issue](https://cloud.githubusercontent.com/assets/6797571/22028163/4e128664-dca4-11e6-9963-5feb45c2e1fb.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3291) UI - Upgrade jQuery

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826298#comment-15826298
 ] 

ASF GitHub Bot commented on NIFI-3291:
--

Github user scottyaslan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1408#discussion_r96442571
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
 ---
@@ -710,8 +784,6 @@
 
 
 assets/**/*,
-css/reset.css,
-css/reset.css.gz,
 css/common-ui.css,
 css/common-ui.css.gz,
 css/message-page.css,
--- End diff --

We need to add the slick-nifi-theme here:

css/slick-nifi-theme.css,
css/slick-nifi-theme.css.gz,


> UI - Upgrade jQuery
> ---
>
> Key: NIFI-3291
> URL: https://issues.apache.org/jira/browse/NIFI-3291
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.2.0
>
>
> Also consider upgrading all jQuery plugins too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2881) Allow Database Fetch processor(s) to accept incoming flow files and use Expression Language

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826199#comment-15826199
 ] 

ASF GitHub Bot commented on NIFI-2881:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1407#discussion_r96434505
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
 ---
@@ -115,20 +128,36 @@ public GenerateTableFetch() {
 
 @OnScheduled
 public void setup(final ProcessContext context) {
+// The processor is invalid if there is an incoming connection and 
max-value columns are defined
+if (context.getProperty(MAX_VALUE_COLUMN_NAMES).isSet() && 
context.hasIncomingConnection()) {
+throw new ProcessException("If an incoming connection is 
supplied, no max-value column names may be specified");
--- End diff --

For backwards-compatibility, what do you think about only falling back to 
the column name as the key if the table name property does not contain 
Expression Language? I'm thinking the only use case we should have to support 
from before is if the table name is static (as it was required to be before 
now). This would not allow for incoming flow files to specify via attribute the 
same table name as before, but I don't see that being a high-priority use case 
necessarily. With this approach, currently configured processors would continue 
to work as-is, but once you change to incoming flow files (or use EL), the new 
scheme would apply. Even currently configured processors would automatically 
switch to the new scheme as soon as a new maximum value is observed.


> Allow Database Fetch processor(s) to accept incoming flow files and use 
> Expression Language
> ---
>
> Key: NIFI-2881
> URL: https://issues.apache.org/jira/browse/NIFI-2881
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>
> The QueryDatabaseTable and GenerateTableFetch processors do not allow 
> Expression Language to be used in the properties, mainly because they also do 
> not allow incoming connections. This means if the user desires to fetch from 
> multiple tables, they currently need one instance of the processor for each 
> table, and those table names must be hard-coded.
> To support the same capabilities for multiple tables and more flexible 
> configuration via Expression Language, these processors should have 
> properties that accept Expression Language, and GenerateTableFetch should 
> accept (optional) incoming connections.
> Conversation about the behavior of the processors is welcomed and encouraged. 
> For example, if an incoming flow file is available, do we also still run the 
> incremental fetch logic for tables that aren't specified by this flow file, 
> or do we just do incremental fetching when the processor is scheduled but 
> there is no incoming flow file. The latter implies a denial-of-service could 
> take place, by flooding the processor with flow files and not letting it do 
> its original job of querying the table, keeping track of maximum values, etc.
> This is likely a breaking change to the processors because of how state 
> management is implemented. Currently since the table name is hard coded, only 
> the column name comprises the key in the state. This would have to be 
> extended to have a compound key that represents table name, max-value column 
> name, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1407: NIFI-2881: Added EL support to DB Fetch processors,...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1407#discussion_r96434505
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
 ---
@@ -115,20 +128,36 @@ public GenerateTableFetch() {
 
 @OnScheduled
 public void setup(final ProcessContext context) {
+// The processor is invalid if there is an incoming connection and 
max-value columns are defined
+if (context.getProperty(MAX_VALUE_COLUMN_NAMES).isSet() && 
context.hasIncomingConnection()) {
+throw new ProcessException("If an incoming connection is 
supplied, no max-value column names may be specified");
--- End diff --

For backwards-compatibility, what do you think about only falling back to 
the column name as the key if the table name property does not contain 
Expression Language? I'm thinking the only use case we should have to support 
from before is if the table name is static (as it was required to be before 
now). This would not allow for incoming flow files to specify via attribute the 
same table name as before, but I don't see that being a high-priority use case 
necessarily. With this approach, currently configured processors would continue 
to work as-is, but once you change to incoming flow files (or use EL), the new 
scheme would apply. Even currently configured processors would automatically 
switch to the new scheme as soon as a new maximum value is observed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2861) ControlRate should accept more than one flow file per execution

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826193#comment-15826193
 ] 

ASF GitHub Bot commented on NIFI-2861:
--

Github user jskora commented on the issue:

https://github.com/apache/nifi/pull/1128
  
Closing per commit a3d95dc1582f2edfd7997c5d8a23105e88729d11 by @mosermw .


> ControlRate should accept more than one flow file per execution
> ---
>
> Key: NIFI-2861
> URL: https://issues.apache.org/jira/browse/NIFI-2861
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Joe Skora
>Assignee: Joe Skora
> Fix For: 0.8.0, 1.2.0
>
>
> The {{ControlRate}} processor implements a {{FlowFileFilter}} that returns 
> the {{FlowFileFilter.ACCEPT_AND_TERMINATE}} result if the {{FlowFile}} fits 
> with the rate limit, affectively limiting it to one {{FlowFile}} per 
> {{ConrolRate.onTrigger()}} invocation.  This is a significant bottleneck when 
> processing very large quantities of small files making it unlikely to hit the 
> rate limits.
> It should allow multiple files, perhaps with a configurable maximum, per 
> {{ControlRate.onTrigger()}} invocation by issuing the 
> {{FlowFileFilter.ACCEPT_AND_CONTINUE}} result until the limits are reached.  
> In a preliminary test this eliminated the bottleneck.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2861) ControlRate should accept more than one flow file per execution

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826194#comment-15826194
 ] 

ASF GitHub Bot commented on NIFI-2861:
--

Github user jskora closed the pull request at:

https://github.com/apache/nifi/pull/1128


> ControlRate should accept more than one flow file per execution
> ---
>
> Key: NIFI-2861
> URL: https://issues.apache.org/jira/browse/NIFI-2861
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Joe Skora
>Assignee: Joe Skora
> Fix For: 0.8.0, 1.2.0
>
>
> The {{ControlRate}} processor implements a {{FlowFileFilter}} that returns 
> the {{FlowFileFilter.ACCEPT_AND_TERMINATE}} result if the {{FlowFile}} fits 
> with the rate limit, affectively limiting it to one {{FlowFile}} per 
> {{ConrolRate.onTrigger()}} invocation.  This is a significant bottleneck when 
> processing very large quantities of small files making it unlikely to hit the 
> rate limits.
> It should allow multiple files, perhaps with a configurable maximum, per 
> {{ControlRate.onTrigger()}} invocation by issuing the 
> {{FlowFileFilter.ACCEPT_AND_CONTINUE}} result until the limits are reached.  
> In a preliminary test this eliminated the bottleneck.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1128: NIFI-2861 ControlRate should accept more than one flow fil...

[jskora]

Github user jskora commented on the issue:

https://github.com/apache/nifi/pull/1128
  
Closing per commit a3d95dc1582f2edfd7997c5d8a23105e88729d11 by @mosermw .


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1128: NIFI-2861 ControlRate should accept more than one f...

[jskora]

Github user jskora closed the pull request at:

https://github.com/apache/nifi/pull/1128


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---