Hello
I think the comments around CompressContent are valid and fixable and your
ideas there make a lot of sense.
As you noted in the second part various components have their own built in
compression. These are likely a bridge too far as their config/options/etc
are well embedded into those libraries.
Thanks
On Thu, Nov 24, 2022 at 6:36 PM Matthew Hawkins wrote:
> Hi team,
>
> Whilst giving a first go at adding brotli and zstd to CompressContent I
> noticed a few things that I think we should address.
>
> 1. compression levels are hard-coded as 0-9 in the UI, this prevents
> accessing every level possible under certain compressors, e.g. zstd has 22
> levels, brotli has 15. I hacked it, it's ugly, we should do the "right"
> thing instead.
> - I think this should be addressed by having a compressor register itself
> along with metadata like how many levels it supports to a factory & the UI
> utilise that property in a more dynamic fashion
> - we could also use metadata to let a compressor flag known high cpu /
> high mem situations so the flow Admin can avoid silly mistakes in config or
> the JVM can be hinted at.
>
> 2. Some other processors such as PublishKafka, InvokeHTTP,
> JsonRecordSetWriter, and others implement some compression algorithms
> directly.
> - This made me realise it would IMO be better offered as a Service and
> each of these use cases can have the simple API of using the controller
> service rather than duplicating compression algorithm specific code each
> time the functionality is required.
>
> Am I on the right track? Java isn't my forte and I'm new to contributing to
> NiFi so keen to hear what experienced devs think.
>
> --
> Kind regards,
> Matthew Hawkins
>
