[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-08-11 Thread Mark Payne (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123960#comment-16123960
 ] 

Mark Payne commented on NIFI-3736:
--

I spawned a new JIRA to address the last issue here, that we are not honoring 
the "nifi.content.claim.max.flow.files". Since the PR here does address the 
first part of "nifi.content.claim.max.appendable.size" I think it makes sense 
to just rename this ticket and spawn a new thread for the other (related but 
separate) issue.

> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
> Fix For: 1.4.0
>
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-08-11 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123937#comment-16123937
 ] 

ASF subversion and git services commented on NIFI-3736:
---

Commit d6ebc53076f597c038e81600dccddc5d16b27663 in nifi's branch 
refs/heads/master from [~boardm26]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d6ebc53 ]

NIFI-3736 modify default nifi.content.claim.max.appendable.size in 
nifi.properties to 1 MB


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
> Fix For: 1.4.0
>
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-08-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123935#comment-16123935
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2041
  
Sorry - missed this comment. Nope, no strong intent. Perhaps even just a 
typo. Will merge.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
> Fix For: 1.4.0
>
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-27 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103899#comment-16103899
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2041
  
@markap14  anything moser and I might be overlooking.  This strikes me as a 
no brainer to do.  Was there a strong intent behind 10MB?


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
> Fix For: 1.4.0
>
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-27 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103875#comment-16103875
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

GitHub user mosermw opened a pull request:

https://github.com/apache/nifi/pull/2041

NIFI-3736 modify default nifi.content.claim.max.appendable.size

in nifi.properties to 1 MB

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mosermw/nifi NIFI-3736

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2041.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2041


commit 641fb4fbc2da1942fead30bf74a3d960769a12ff
Author: Mike Moser 
Date:   2017-07-27T20:42:09Z

NIFI-3736 modify default nifi.content.claim.max.appendable.size in 
nifi.properties to 1 MB




> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
> Fix For: 1.4.0
>
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-27 Thread Joseph Witt (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103840#comment-16103840
 ] 

Joseph Witt commented on NIFI-3736:
---

Both seem like good points to me. 

#1 It makes sense that we ensure both max number of claims and max total size 
of claims can be tuned.
#2 I agree.  We should at least be consistent.  The smaller 1MB value sounds 
about right.  We're just trying to help make writes/reads more efficient.  
Going much larger likely wont help much anyway.

> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
> Fix For: 1.4.0
>
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-27 Thread Michael Moser (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103804#comment-16103804
 ] 

Michael Moser commented on NIFI-3736:
-

While building a template to show the effect of NIFI-3376, I found 2 things 
about this issue.

1. The nifi.content.claim.max.flow.files property setting has no effect on 
claim size.  Shall I reopen this JIRA to address this or make a new one?
2. In 1.3.0, the effective nifi.content.claim.max.appendable.size was hard 
coded to 1 MB.  Now that the property is used, the default for that property is 
10 MB.  This is a pretty significant change to default behavior of the content 
repository.  I think we should set the default for this property to 1 MB to 
match 1.3.0 behavior.  Shall I make a new JIRA to address this?

> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
> Fix For: 1.4.0
>
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096260#comment-16096260
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2010


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096258#comment-16096258
 ] 

ASF subversion and git services commented on NIFI-3736:
---

Commit c54b2ad81c07e96f26d1ef19d0f887a0f7704da5 in nifi's branch 
refs/heads/master from m-hogue
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c54b2ad ]

NIFI-3736: change to honor nifi.content.claim.max.appendable.size and 
nifi.content.claim.max.flow.files properties. Added 100 MB cap for 
NiFiProperties.MAX_APPENDABLE_CLAIM_SIZE

This closes #2010.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096259#comment-16096259
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2010
  
@m-hogue thanks for the update! +1 merged to master.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095465#comment-16095465
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user m-hogue commented on the issue:

https://github.com/apache/nifi/pull/2010
  
@markap14 @mosermw : I've added a static 100MB cap for the max appendable 
claim size. Please let me know if you'd like any more changes.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095462#comment-16095462
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user m-hogue commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2010#discussion_r128642452
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
 ---
@@ -149,6 +154,10 @@ public FileSystemRepository(final NiFiProperties 
nifiProperties) throws IOExcept
 for (final Path path : fileRespositoryPaths.values()) {
 Files.createDirectories(path);
 }
+this.maxFlowFilesPerClaim = 
nifiProperties.getMaxFlowFilesPerClaim();
+this.writableClaimQueue  = new 
LinkedBlockingQueue<>(maxFlowFilesPerClaim);
+final String maxAppendableClaimSize = 
nifiProperties.getMaxAppendableClaimSize();
+this.maxAppendableClaimLength = 
DataUnit.parseDataSize(maxAppendableClaimSize, DataUnit.B).intValue();
--- End diff --

Cool - I'll add a static 100MB cap.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095388#comment-16095388
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user mosermw commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2010#discussion_r128633148
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
 ---
@@ -149,6 +154,10 @@ public FileSystemRepository(final NiFiProperties 
nifiProperties) throws IOExcept
 for (final Path path : fileRespositoryPaths.values()) {
 Files.createDirectories(path);
 }
+this.maxFlowFilesPerClaim = 
nifiProperties.getMaxFlowFilesPerClaim();
+this.writableClaimQueue  = new 
LinkedBlockingQueue<>(maxFlowFilesPerClaim);
+final String maxAppendableClaimSize = 
nifiProperties.getMaxAppendableClaimSize();
+this.maxAppendableClaimLength = 
DataUnit.parseDataSize(maxAppendableClaimSize, DataUnit.B).intValue();
--- End diff --

In my opinion, I don't see anything wrong with a static cap, as long as 
it's reasonably big enough (the 100 MB choice).


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095312#comment-16095312
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user m-hogue commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2010#discussion_r128622791
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
 ---
@@ -149,6 +154,10 @@ public FileSystemRepository(final NiFiProperties 
nifiProperties) throws IOExcept
 for (final Path path : fileRespositoryPaths.values()) {
 Files.createDirectories(path);
 }
+this.maxFlowFilesPerClaim = 
nifiProperties.getMaxFlowFilesPerClaim();
+this.writableClaimQueue  = new 
LinkedBlockingQueue<>(maxFlowFilesPerClaim);
+final String maxAppendableClaimSize = 
nifiProperties.getMaxAppendableClaimSize();
+this.maxAppendableClaimLength = 
DataUnit.parseDataSize(maxAppendableClaimSize, DataUnit.B).intValue();
--- End diff --

Yep - that makes total sense. Should the max be configurable or static? I 
can do either way.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095235#comment-16095235
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2010
  
@m-hogue thanks for submitting the PR! This has been on my to-do list for a 
long time, and I'm very happy to have you knocking it out. The only concern 
that I have is the one that i mentioned inline, regarding what will happen if a 
user sets a very large value for the max.appendable.claim.size. Otherwise, I 
think all looks good.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095233#comment-16095233
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2010#discussion_r128609849
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
 ---
@@ -149,6 +154,10 @@ public FileSystemRepository(final NiFiProperties 
nifiProperties) throws IOExcept
 for (final Path path : fileRespositoryPaths.values()) {
 Files.createDirectories(path);
 }
+this.maxFlowFilesPerClaim = 
nifiProperties.getMaxFlowFilesPerClaim();
+this.writableClaimQueue  = new 
LinkedBlockingQueue<>(maxFlowFilesPerClaim);
+final String maxAppendableClaimSize = 
nifiProperties.getMaxAppendableClaimSize();
+this.maxAppendableClaimLength = 
DataUnit.parseDataSize(maxAppendableClaimSize, DataUnit.B).intValue();
--- End diff --

If this value gets set to something like "10 GB" this could cause some 
really problematic (and difficult to track down) problems because the value 
would overflow to a negative value. Probably is best to use a longValue() and 
then perhaps even cap it at something like 100 MB or 10 MB and if the value is 
larger than that, just emit a WARN log event and use the max value.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087577#comment-16087577
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

Github user m-hogue commented on the issue:

https://github.com/apache/nifi/pull/2010
  
This should resolve https://issues.apache.org/jira/browse/NIFI-1603 as well.


> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087573#comment-16087573
 ] 

ASF GitHub Bot commented on NIFI-3736:
--

GitHub user m-hogue opened a pull request:

https://github.com/apache/nifi/pull/2010

NIFI-3736: change to honor nifi.content.claim.max.appendable.size and…

… nifi.content.claim.max.flow.files properties

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/m-hogue/nifi NIFI-3736

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2010.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2010


commit b8111e497075b8ff31560c7bbc53271a8503f1f6
Author: m-hogue 
Date:   2017-07-14T16:50:45Z

NIFI-3736: change to honor nifi.content.claim.max.appendable.size and 
nifi.content.claim.max.flow.files properties




> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-07-14 Thread Michael Hogue (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087572#comment-16087572
 ] 

Michael Hogue commented on NIFI-3736:
-

Work here: https://github.com/m-hogue/nifi/tree/NIFI-3736

> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Michael Hogue
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3736) NiFi not honoring the "nifi.content.claim.max.appendable.size" and "nifi.content.claim.max.flow.files" properties

2017-04-26 Thread Michael Moser (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984996#comment-15984996
 ] 

Michael Moser commented on NIFI-3736:
-

Duplicate of NIFI-1603.

> NiFi not honoring the "nifi.content.claim.max.appendable.size" and 
> "nifi.content.claim.max.flow.files" properties
> -
>
> Key: NIFI-3736
> URL: https://issues.apache.org/jira/browse/NIFI-3736
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>
> The nifi.properties file has two properties for controlling how many 
> FlowFiles to jam into one Content Claim. Unfortunately, it looks like this is 
> no longer honored in FileSystemRepository.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)