Thanks Lee.  Your response was awesome and really made me want to get
hands on a set of boxes like this so we could do some testing.

Thanks
Joe

On Mon, Oct 17, 2016 at 11:32 AM, Lee Laim <lee.l...@gmail.com> wrote:
> Joe,
> Good points regarding throughput on real flows and sustained basis.  My test
> was only pushing one aspect of the system.
>
> That said, I would be interested discussing/developing a more comprehensive
> test flow to capture more real world use cases. I'll check to see if that
> conversation has started.
>
> Thanks,
> Lee
>
>
>
>
>
> Lee Laim
> 610-864-1657
>
> On Oct 17, 2016, at 9:55 PM, Ali Nazemian <alinazem...@gmail.com> wrote:
>
> Dear Joe,
> Thank you very much.
>
> Best regards
>
>
> On Mon, Oct 17, 2016 at 10:08 PM, Joe Witt <joe.w...@gmail.com> wrote:
>>
>> Ali
>>
>> I suspect bottlenecks in the software itself and the flow design will
>> become a factor before you 800 MB/s. You'd likely hit CPU efficiency
>> issues before this caused by the flow processors themselves and due to
>> garbage collection.  Probably the most important factor though will be
>> the transaction rate and whether the flow is configured to tradeoff
>> some latency for higher throughput.  So many variables at play but
>> under idealized conditions and a system like you describe it is
>> theoretically feasible to hit that value.
>>
>> Practically speaking I think you'd be looking at a couple hundred MB/s
>> per server like this on real flows on a sustained basis.
>>
>> Thanks
>> Joe
>>
>> On Sun, Oct 16, 2016 at 11:06 PM, Ali Nazemian <alinazem...@gmail.com>
>> wrote:
>> > Dear Nifi users/developers,
>> > Hi,
>> >
>> > I was wondering how can I calculate the theoretical throughput of a Nifi
>> > server? let's suppose we can eliminate different bottlenecks such as the
>> > file flow rep and provenance repo bottleneck by using a very high-end
>> > SSD.
>> > Moreover, assume that a very high-end network infrastructure is
>> > available.
>> > In this case, is it possible to reach 800MB throughput per second per
>> > each
>> > server? Suppose each server comes with 24 disk slots. 16 disk slots are
>> > used
>> > for creating 8 x RAID1(SAS 10k) mount points and are dedicated to the
>> > content repo. Let's say each content repo can achieve 100 MB throughput.
>> > May
>> > I say the total throughput per each server can be 8x100=800MBps?  Is it
>> > possible to reach this amount of throughput practically?
>> > Thank you very much.
>> >
>> > Best regards,
>> > Ali
>
>
>
>
> --
> A.Nazemian

Reply via email to