Re: Expression language access to flowfile content?
Thanks for the explanation!
On Mon, Jan 16, 2017 at 12:24 PM, Joe Percivall <jperciv...@apache.org>
wrote:
> Nick,
>
> A main reason why content isn't referenceable by expression language is
> that it promotes an anti-pattern. Reading the content takes significantly
> more time than attributes and should be limited as much as possible. Also
> content can potentially be very large and reading the entire thing in
> memory can cause many adverse effects.
>
> Having less processors does not mean the flow is better (similar to
> concise code != efficient code). You can have a couple very specialized
> processors that very effectively stream the content and do operations as
> needed. Alternatively using EL to grab the content each time ${content()}
> is called would introduce many processors worth of memory and IO
> inefficiency.
>
> You can learn more about the different repositories by reading this doc[1].
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html
>
> Joe
>
> On Mon, Jan 16, 2017 at 3:15 PM, Nick Carenza <
> nick.care...@thecontrolgroup.com> wrote:
>
>> I found a section in the nifi expression language documentation of
>> Subjectless Functions. Could/should this feature be added as one of those?
>>
>> On Mon, Jan 16, 2017 at 11:53 AM, Nick Carenza <
>> nick.care...@thecontrolgroup.com> wrote:
>>
>>> Thanks Bryan, that is what I ended up doing. PutEmail works the same was
>>> as PutSlack in this regard. It expects you to specify a message as a
>>> property that supports expression language and doesn't give you an option
>>> to use the flowfile's content in the message but _does_ allow you to attach
>>> the flowfile.
>>>
>>> If expression language was capable of retrieving flowfile content
>>> directly, I could reduce the number of processors significantly.
>>>
>>> A major problem with this might be the need to introduce a reserved
>>> attribute which would have the potential to cause compatibility problems in
>>> some flows. Or perhaps a function that could serve as the root of an
>>> expression `${content()}`.
>>>
>>> On Mon, Jan 16, 2017 at 11:26 AM, Bryan Bende <bbe...@gmail.com> wrote:
>>>
>>>> Nick,
>>>>
>>>> The current approach is to use ExtractText to extract the entire flow
>>>> file content to an attribute which can then be referenced in expression
>>>> language.
>>>>
>>>> Keep in mind this means the entire content will be read into memory
>>>> which in some cases may not be a good idea.
>>>>
>>>> I would think that PutSlack should have a strategy to decide where the
>>>> message should come from (attribute vs content), but I am not familiar with
>>>> that processor to really say if it is a good idea.
>>>>
>>>> -Bryan
>>>>
>>>>
>>>> On Mon, Jan 16, 2017 at 1:40 PM, Nick Carenza <
>>>> nick.care...@thecontrolgroup.com> wrote:
>>>>
>>>>> Is there any way to access flowfile content with expression language?
>>>>>
>>>>> I am trying to use monitor activity with putslack but monitor activity
>>>>> creates flowfiles with confiugrable content but putslack requires you to
>>>>> supply a message property using expression language which as far as i can
>>>>> tell doesn't have access to that flowfile content.
>>>>>
>>>>> Without having to put another processor in between monitor activity
>>>>> and put slack, is there a way to use the flowfile content directly from
>>>>> expression language?
>>>>>
>>>>> If not does anyone else think this would be really useful to make
>>>>> processors like these more compatible?
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> *Joe Percivall*
> linkedin.com/in/Percivall
> e: jperciv...@apache.com
>
Re: Expression language access to flowfile content?
Nick,
A main reason why content isn't referenceable by expression language is
that it promotes an anti-pattern. Reading the content takes significantly
more time than attributes and should be limited as much as possible. Also
content can potentially be very large and reading the entire thing in
memory can cause many adverse effects.
Having less processors does not mean the flow is better (similar to concise
code != efficient code). You can have a couple very specialized processors
that very effectively stream the content and do operations as needed.
Alternatively using EL to grab the content each time ${content()} is called
would introduce many processors worth of memory and IO inefficiency.
You can learn more about the different repositories by reading this doc[1].
[1] https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html
Joe
On Mon, Jan 16, 2017 at 3:15 PM, Nick Carenza <
nick.care...@thecontrolgroup.com> wrote:
> I found a section in the nifi expression language documentation of
> Subjectless Functions. Could/should this feature be added as one of those?
>
> On Mon, Jan 16, 2017 at 11:53 AM, Nick Carenza <
> nick.care...@thecontrolgroup.com> wrote:
>
>> Thanks Bryan, that is what I ended up doing. PutEmail works the same was
>> as PutSlack in this regard. It expects you to specify a message as a
>> property that supports expression language and doesn't give you an option
>> to use the flowfile's content in the message but _does_ allow you to attach
>> the flowfile.
>>
>> If expression language was capable of retrieving flowfile content
>> directly, I could reduce the number of processors significantly.
>>
>> A major problem with this might be the need to introduce a reserved
>> attribute which would have the potential to cause compatibility problems in
>> some flows. Or perhaps a function that could serve as the root of an
>> expression `${content()}`.
>>
>> On Mon, Jan 16, 2017 at 11:26 AM, Bryan Bende <bbe...@gmail.com> wrote:
>>
>>> Nick,
>>>
>>> The current approach is to use ExtractText to extract the entire flow
>>> file content to an attribute which can then be referenced in expression
>>> language.
>>>
>>> Keep in mind this means the entire content will be read into memory
>>> which in some cases may not be a good idea.
>>>
>>> I would think that PutSlack should have a strategy to decide where the
>>> message should come from (attribute vs content), but I am not familiar with
>>> that processor to really say if it is a good idea.
>>>
>>> -Bryan
>>>
>>>
>>> On Mon, Jan 16, 2017 at 1:40 PM, Nick Carenza <
>>> nick.care...@thecontrolgroup.com> wrote:
>>>
>>>> Is there any way to access flowfile content with expression language?
>>>>
>>>> I am trying to use monitor activity with putslack but monitor activity
>>>> creates flowfiles with confiugrable content but putslack requires you to
>>>> supply a message property using expression language which as far as i can
>>>> tell doesn't have access to that flowfile content.
>>>>
>>>> Without having to put another processor in between monitor activity and
>>>> put slack, is there a way to use the flowfile content directly from
>>>> expression language?
>>>>
>>>> If not does anyone else think this would be really useful to make
>>>> processors like these more compatible?
>>>>
>>>
>>>
>>
>
--
*Joe Percivall*
linkedin.com/in/Percivall
e: jperciv...@apache.com
Expression language access to flowfile content?
Is there any way to access flowfile content with expression language? I am trying to use monitor activity with putslack but monitor activity creates flowfiles with confiugrable content but putslack requires you to supply a message property using expression language which as far as i can tell doesn't have access to that flowfile content. Without having to put another processor in between monitor activity and put slack, is there a way to use the flowfile content directly from expression language? If not does anyone else think this would be really useful to make processors like these more compatible?
