Hi Joe,
Thanks for the explanations. Really useful in understanding how it works.
Good to know that in the future this will be improved.
About the appending to HDFS issue let me recap. My flow is:
ListHDFS -> FetchHDFS -> UnpackContent -> SplitText(5000) -> SplitText(1)
-> RouteOnContent -> Merge
Hello Roman,
It seems the resolution of last modified timestamp depends on the file
system implementation.
https://stackoverflow.com/questions/3805201/how-to-get-ubuntu-file-timestamp-in-millisecond
I reproduced the same behavior on OS X, which uses HFS that has the
same limitation of resolution
Thanks everyone! Very grateful to be a part of this awesome project and
community.
-Drew
> On May 31, 2017, at 11:46 PM, Joe Percivall wrote:
>
> Congrats Drew! Very well deserved, always appreciated all your work on
> documentation and QA
>
> On Wed, May 31, 2017 at 10:49 PM, Koji Kawamura
Congrats Marc!
> On May 31, 2017, at 11:45 PM, Joe Percivall wrote:
>
> Congrats Marc, excited to see what else you can create for MiNiFi!
>
> On Wed, May 31, 2017 at 11:00 PM, Joe Witt wrote:
>
>> Congrats and welcome. Kevin said it very well so I'll simply +1 that.
>>
>> On Wed, May 31,
Congrats Drew! Very well deserved, always appreciated all your work on
documentation and QA
On Wed, May 31, 2017 at 10:49 PM, Koji Kawamura
wrote:
> Congratulations Drew!!
>
> On Thu, Jun 1, 2017 at 11:47 AM, Yolanda Davis
> wrote:
> >
> > Congratulations Drew!
> >
> >
> >> On May 31, 2017, at
Congrats Marc, excited to see what else you can create for MiNiFi!
On Wed, May 31, 2017 at 11:00 PM, Joe Witt wrote:
> Congrats and welcome. Kevin said it very well so I'll simply +1 that.
>
> On Wed, May 31, 2017 at 10:45 PM, Kevin Doran
> wrote:
> >
> >
> >
> >
> >
> >
> > Co
Congrats and welcome. Kevin said it very well so I'll simply +1 that.
On Wed, May 31, 2017 at 10:45 PM, Kevin Doran wrote:
>
>
>
>
>
>
> Congratulations, Marc! Very well deserved. Thanks for your
> contributions and generosity with sharing your technical knowledge.
>
>
>
>
Congratulations Drew!!
On Thu, Jun 1, 2017 at 11:47 AM, Yolanda Davis
wrote:
>
> Congratulations Drew!
>
>
>> On May 31, 2017, at 4:20 PM, Bryan Rosander wrote:
>>
>> Congrats Drew!
>>
>> On Wed, May 31, 2017 at 3:51 PM, Andrew Psaltis
>> wrote:
>>
>>> Congrats Drew! I spend a lot of time teach
Congratulations Drew!
> On May 31, 2017, at 4:20 PM, Bryan Rosander wrote:
>
> Congrats Drew!
>
> On Wed, May 31, 2017 at 3:51 PM, Andrew Psaltis
> wrote:
>
>> Congrats Drew! I spend a lot of time teaching people about NiFi and can
>> honestly say the documentation -- built-in and on the wi
Congratulations, Marc! Very well deserved. Thanks for your
contributions and generosity with sharing your technical knowledge.
_
From: Jeremy
Congratulations Marc! Have really enjoyed working and learning from you on the
C++ side of the house. Look forward to your further work!
Sent from my iPhone
> On May 31, 2017, at 9:37 PM, Tony Kurc wrote:
>
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Marc
> Parisi ha
On behalf of the Apache NiFI PMC, I am very pleased to announce that Marc
Parisi has accepted the PMC's invitation to become a committer on the
Apache NiFi project. We greatly appreciate all of Marc's hard work and
generous contributions to the project. We look forward to his continued
involvement
Congrats Drew!
On Wed, May 31, 2017 at 3:51 PM, Andrew Psaltis
wrote:
> Congrats Drew! I spend a lot of time teaching people about NiFi and can
> honestly say the documentation -- built-in and on the wiki is a great
> selling point. Thanks for all the hard work.
>
> On Wed, May 31, 2017 at 3:08
Congrats Drew! I spend a lot of time teaching people about NiFi and can
honestly say the documentation -- built-in and on the wiki is a great
selling point. Thanks for all the hard work.
On Wed, May 31, 2017 at 3:08 PM, Tony Kurc wrote:
> On behalf of the Apache NiFI PMC, I am very pleased to an
JoeG: If you or anyone else happens to get in that state again please
enable DEBUG level logging for
"org.apache.nifi.controller.StandardFlowFileQueue". Changes to your
logback configuration will get picked up without needing to restart.
Thanks!
Matt
On Wed, May 31, 2017 at 3:23 PM, Joe Witt wr
JoeG: Anything you can do to make that reproducible would be key. So
far it is been unreproducible.
MattG : Huge thanks for taking on RM for these. I'll knock out
reviews on markp's prs for 1.3.0. Three of them there that look easy
to tackle.
Thanks!
On Wed, May 31, 2017 at 3:19 PM, Joe Greso
Just wanted to put in a request for anyone to look at [1] before the 1.3.0
release, if possible. It's a tough one because it's hard to reproduce, but
I feel like its cause could be deduced by inspecting any changes in the
flowfile/content repositories between version 1.1.1 and 1.2.0 (since the
pro
Hi again,
Just wanted to follow up with the current listing of JIRAs for the 1.3.0
release [1] and the 0.7.4 release [2] for folks who are interested.
Thanks
Matt
[1] https://issues.apache.org/jira/browse/NIFI/fixforversion/12340498
[2] https://issues.apache.org/jira/browse/NIFI/fixforversion/1
Big thanks for all you've done to help shape the ux and docs within
NiFi and congrats on committership!
On Wed, May 31, 2017 at 3:10 PM, Pierre Villard
wrote:
> Congrats Drew, really pleased to have you on board, you're doing a great
> job!
>
> 2017-05-31 21:08 GMT+02:00 Tony Kurc :
>
>> On beh
Congrats Drew, really pleased to have you on board, you're doing a great
job!
2017-05-31 21:08 GMT+02:00 Tony Kurc :
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Drew
> Lim has accepted the PMC's invitation to become a committer on the Apache
> NiFi project. We greatly a
On behalf of the Apache NiFI PMC, I am very pleased to announce that Drew
Lim has accepted the PMC's invitation to become a committer on the Apache
NiFi project. We greatly appreciate all of Drew's hard work and generous
contributions to the project. We look forward to his continued involvement
in
Hi,
There's been a number great bug fixes and features added since the release
of 1.2.0. Many of these bug fixes also were relevant to the 0.x release
line. I am going to start getting everything together with the hopes of
putting out release candidates for 1.3.0 and 0.7.4 and their respective
VOT
Hi there, i need help.
We prepare high load project and tested this processors. All time see
listing.timestamp and processed.timestamp keys without milliseconds
(xx000). In this way, if generate several files in one second, not
all files will be listened.
Test:
1. start processor ListFil
Split failed before even with backpressure:
- yes that backpressure kicks in when destination queues for a given
processor have reached their target size (in count of flowfiles or
total size represented). However, to clarify why the OOM happened it
is important to realize that it is not about 'flo
Hi Koji,
Good to know that it can handle large files. I thought it was the case but
I was just not seeing in practice.
Yes I am using 'Line Split Count' as 1 at SplitText.
I added the extra SplitText processor exactly as you suggested and the OOM
went away. So, big thanks!!!
However I have 2 fo
Hi Martin,
Generally, NiFi processor doesn't load entire content of file and is
capable of handling huge files.
However, having massive amount of FlowFiles can cause OOM issue as
FlowFiles and its Attributes resides on heap.
I assume you are using 'Line Split Count' as 1 at SplitText.
We recommen
Hi all,
I have a vanilla Nifi 1.2.0 node with 1GB of heap.
The flow I am trying to run is:
ListHDFS -> FetchHDFS -> SplitText -> RouteOnContent -> MergeContent ->
PutHDFS
When I give it a 300MB input zip file (2.5GB uncompressed) I am getting
Java OutOfMemoryError as below.
Does NiFi read in th
27 matches
Mail list logo