e Witt
> Sent: Friday, 14 September 2018 1:37 PM
> To: dev@nifi.apache.org
> Subject: Re: Ideal hardware for NiFi
>
> The ff disk needs to be the quickest disk and should have no other
> contention just like a db trans log would request.
>
> The prov repo should also have
Witt
Sent: Friday, 14 September 2018 1:37 PM
To: dev@nifi.apache.org
Subject: Re: Ideal hardware for NiFi
The ff disk needs to be the quickest disk and should have no other
contention just like a db trans log would request.
The prov repo should also have its pwn disk.
The content repo can have
The ff disk needs to be the quickest disk and should have no other
contention just like a db trans log would request.
The prov repo should also have its pwn disk.
The content repo can have one or more physical disks.
The best case is each repo is on physically separate disks/underlying
storage.
Hi joe,
I moved the content and providence repositories off to two new disks, but
it seems like the vast majority of the writes are still occurring on the
disk where the flowfile and database repositories are. I note they don't
appear to be able to be split across disks in the same way?
On Fri,
if they are physically seperate the diff should be quite noticable.
On Thu, Sep 13, 2018, 7:36 PM Phil H wrote:
> Potentially. We're looking to see how the multiple disks help before
> committing to spending money on new hardware :)
>
> On Fri, 14 Sep 2018 at 10:48, Joe Witt wrote:
>
> > phil,
Potentially. We're looking to see how the multiple disks help before
committing to spending money on new hardware :)
On Fri, 14 Sep 2018 at 10:48, Joe Witt wrote:
> phil,
>
> as you add dirs it will just start using them. if you want to no longer
> use the current dir it might be more
phil,
as you add dirs it will just start using them. if you want to no longer
use the current dir it might be more involved.
does that help?
thanks
On Thu, Sep 13, 2018, 4:36 PM Phil H wrote:
> Follow up question - how do I transition to this new structure? Should I
> shut down NiFi and
Follow up question - how do I transition to this new structure? Should I
shut down NiFi and move the contents of the legacy single directories into
one of the new ones? For example:
mv /usr/nifi/content_repository
/nifi/repos/content-1
TIA
Phil
On Wed, 12 Sep 2018 at 06:15, Mark Payne wrote:
Fantastic, thanks Mark
On Wed, 12 Sep 2018 at 06:15, Mark Payne wrote:
> Phil,
>
> For the content repository, you can configure the directory by changing
> the value of
> the "nifi.content.repository.directory.default" property in
> nifi.properties. The suffix here,
> "default" is the name of
Phil,
For the content repository, you can configure the directory by changing the
value of
the "nifi.content.repository.directory.default" property in nifi.properties.
The suffix here,
"default" is the name of this "container". You can have multiple containers by
adding extra
properties. So,
Thanks Mark, this is great advice.
Disk access is certainly an issue with the current set up. I will certainly
shoot for NVMe disks in the build. How does NiFi get configured to span
it's repositories across multiple physical disks?
Thanks,
Phil
On Wed, 12 Sep 2018 at 01:32, Mark Payne wrote:
Phil,
As Sivaprasanna mentioned, your bottleneck will certainly depend on your flow.
There's nothing inherent about NiFi or the JVM, AFAIK that would limit you. I've
seen NiFi run on VM's containing 4-8 cores, and I've seen it run on bare metal
on servers containing 96+ cores. Most often, I see
Thanks for that. Sorry I should have been more specific - we have a flow
running already on non-dedicated hardware. Looking to identify any
limitations in NiFi/JVM that would limit how much parallelism it can take
advantage of
On Mon, 10 Sep 2018 at 14:32, Sivaprasanna
wrote:
> Phil,
>
> The
13 matches
Mail list logo
