Currently variables are not encrypted or protected sufficiently, so this
behavior is discouraged and you should at least be aware of those risks. There
are numerous discussions, usually on PRs, where we explain that there are
difficulties in protecting those values and parsing EL in sensitive properties
— for example, a password “abc${def}”. Is that a literal 9 character password,
or should the value of variable “def” be appended to the literal “abc”?
Andy LoPresto
[email protected]
[email protected]
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Oct 27, 2018, at 04:40, Jon Logan <[email protected]> wrote:
>
> Speaking of EL and Secrets -- we have been setting some secret values as EL
> expressions to substitute in properties for passwords and such. If we
> switch to NiFi registry, this would still be considered a sensitive value
> and not be stored? Is there a better way to handle secret distribution?
>
>> On Fri, Oct 26, 2018 at 3:38 PM Michael Moser <[email protected]> wrote:
>>
>> For the specific case of "host name of the AMQP processors", EL support for
>> that property was added to NiFi in NIFI-5489 [1] which will be released
>> with NiFi 1.8.0 (available real soon!)
>>
>> [1] - https://issues.apache.org/jira/browse/NIFI-5489
>>
>> -- Mike
>>
>>
>> On Wed, Oct 24, 2018 at 3:06 PM Pierre Villard <
>> [email protected]>
>> wrote:
>>
>>> Hi there,
>>>
>>> There is a JIRA [1] about this idea and, as Joe said, we need to
>> carefully
>>> think about the implications. Unless we can do something nice and
>> backward
>>> compatible, that could be something for NiFi 2.0.
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-5367
>>>
>>> Pierre
>>>
>>>> Le mer. 24 oct. 2018 à 20:59, Joe Witt <[email protected]> a écrit :
>>>>
>>>> Jon,
>>>>
>>>> There is very little overhead really. The answer more generally has
>>>> to do with history. Expression language was added later in the game
>>>> and there can be different scopes to what a given EL statement has
>>>> access to. Some EL are evaluated against flowfiles and some arent.
>>>> We didn't have a good way to show users this information though this
>>>> has recently improved. It also has to do with 'evaluation time'
>>>> meaning processors might evaluate some properties frequently/while
>>>> executing a task (as would be the case when EL statements have a given
>>>> flowfile in scope) or during enabling/scheduling such as pulling a
>>>> property value they intend to honor throughout an entire runtime.
>>>>
>>>> I suspect at this point that we can make most properties (not
>>>> enumerations most likely) EL enabled. But, each one requires
>>>> review/consideration to ensure the processor is coded to use it
>>>> correctly.
>>>>
>>>> Alternatively, we could explore making all text entry fields allow
>>>> users to explicitly state 'this is a variable and here is the variable
>>>> name' (but in a cool UI way) and then the actual value used would come
>>>> from the variable registry for that process group. This would be
>>>> something we can do across the board as it wouldnt' change the
>>>> processor's logic/behavior at all. Variables get set at and retained
>>>> throughout the life of a processor run. If a used variable is changed
>>>> we'd automatically stop/start each impacted component/processor so it
>>>> wouldnt' require any additional review/changes in processor
>>>> properties.
>>>>
>>>> ThanksOn Wed, Oct 24, 2018 at 2:46 PM Jonathan Meran
>>>> <[email protected]> wrote:
>>>>>
>>>>> Hello there,
>>>>> Regarding expression language properties, why aren't all or most
>>>> properties enabled by default.
>>>>>
>>>>> Is there substantial processor overhead on these?
>>>>>
>>>>> Thanks,
>>>>> Jon
>>>>>
>>>>> On 10/24/18, 1:49 PM, "Joe Witt" <[email protected]> wrote:
>>>>>
>>>>> Mark
>>>>>
>>>>> There are two scenarios here to discuss:
>>>>> 1) What do to with sensitive properties
>>>>> 2) What to do with properties that dont allow expression language
>>>> statements
>>>>>
>>>>> For #1 we dont send those properties into the registry and they
>> are
>>>>> set (properly) in each environment where the secret belongs. All
>>>> good
>>>>> as that doesn't change the flow definition.
>>>>>
>>>>> For #2 we should just look at which properties are causing
>> trouble
>>>> and
>>>>> see about expression language enabling them. So please share
>>>>> precisely which ones you're hitting where that would help you out
>>> and
>>>>> lets see what can be done.
>>>>>
>>>>> Thanks
>>>>> On Wed, Oct 24, 2018 at 1:39 PM Mark Littleton
>>>>> <[email protected]> wrote:
>>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> I'm currently doing a lot of work with Nifi and recently we
>> have
>>>> been trying to come up with a solution to a problem. We installed Nifi
>>>> registry backed by our Git repository for versioning our flows. This
>> has
>>>> worked out great for us as we can now track the version of our flows
>>>> correctly and make sure they are backed up in source control.
>>>>>>
>>>>>> However when we want to do deployment between our Development
>>> Nifi
>>>> cluster and our Qa Nifi cluster we have to ofcourse change some values.
>>>> These could be amqp queues, directories on the file system etc.
>>>>>>
>>>>>> So ofcourse we use variables so that we can configure the
>> values
>>>> without it being detected as a change to the flow. A problem arises
>>> however
>>>> when we need to configure an option that does not support expression
>>>> language. For example the host name of the amqp processors.
>>>>>>
>>>>>> This leaves us in a situation where a change to the flow is
>>>> detected. The only real option I have as far as I can see is to clone
>> the
>>>> flows and have one for each environment which I don't like at all.
>>>>>>
>>>>>> Is anyone else struggling with similar issues. If so how are
>> you
>>>> handling It?
>>>>>>
>>>>>> Sent from my Sony Xperia™ smartphone
>>>>>
>>>>>
>>>>
>>>
>>
