Re: exporting activation arguments to the environment
I opened issue https://github.com/apache/incubator-openwhisk/issues/4558 and corresponding PR https://github.com/apache/incubator-openwhisk/pull/4559 . Additional feedback and critique welcomed. -r
Re: exporting activation arguments to the environment
I implemented both, the "convention" and the parameters designated as init through the API. The latter wasn't much more over the former and is backward compatible. This can be introduced incrementally into the main repo/REST API, then the runtimes (I added it to Node but not the others), and eventually the CLI and other tooling to introduce a "-e". > - this is also a bigger convention change for action developers (read from env or context instead of the single args object given to main) We have no mechanism today for developers to use env vars with their actions. This new capability actually leads to nicer code -- one prime example is the use of 'npm openwhisk' which can now be lifted to a top level constant that's automatically initialized, or using libraries such as sequelize in a similar way. So while it would require developers to take advantage of this, it doesn't break any existing code. On Thu, Jun 27, 2019 at 1:25 PM Tyson Norris wrote: > BTW, circling back on "can it be done just using -p in current form": I > still like the idea that: > - action configured params are different the user specified params (and we > should only pass them on init, not run) > - this is also a bigger convention change for action developers (read from > env or context instead of the single args object given to main) > > But - *adding* the -e, instead of revising the meaning of -p does makes > this a backwards compatible change, so that's a good thing. > > Thanks > Tyson > > >
Re: exporting activation arguments to the environment
BTW, circling back on "can it be done just using -p in current form": I still like the idea that: - action configured params are different the user specified params (and we should only pass them on init, not run) - this is also a bigger convention change for action developers (read from env or context instead of the single args object given to main) But - *adding* the -e, instead of revising the meaning of -p does makes this a backwards compatible change, so that's a good thing. Thanks Tyson On 6/27/19, 5:00 AM, "Rodric Rabbah" wrote: Thanks for the added feedback - keep it coming! Tyson is right in that we shouldn't let implementation concerns affect a proper design and sacrifcie the experience. So in line with his concerns, is there a desire to facilitate environment variables at all (user specified values, not system/context ones)? If so, then having thought about it some more, annotating every parameter could be done as suggested say with a -e vs a -p. -r
Re: exporting activation arguments to the environment
Based on the giant mess in Jenkins where parameters and environment variables sometimes get conflated, I'd definitely recommend making them separate, especially for inevitable security issues found related to them. ;) On Thu, 27 Jun 2019 at 07:00, Rodric Rabbah wrote: > > Thanks for the added feedback - keep it coming! > > Tyson is right in that we shouldn't let implementation concerns affect a > proper design and sacrifcie the experience. > > So in line with his concerns, is there a desire to facilitate environment > variables at all (user specified values, not system/context ones)? > > If so, then having thought about it some more, annotating every parameter > could be done as suggested say with a -e vs a -p. > > -r -- Matt Sicker
Re: exporting activation arguments to the environment
Thanks for the added feedback - keep it coming! Tyson is right in that we shouldn't let implementation concerns affect a proper design and sacrifcie the experience. So in line with his concerns, is there a desire to facilitate environment variables at all (user specified values, not system/context ones)? If so, then having thought about it some more, annotating every parameter could be done as suggested say with a -e vs a -p. -r
Re: exporting activation arguments to the environment
Hi, In general, I'd rather see a different switch such as `-e` for setting environment variables as I'm suspicious of a `param` having different meanings which risks confusion. Regards, Rob > On 27 Jun 2019, at 01:21, Tyson Norris wrote: > > For the incremental change I would suggest: > - include the action-configured params as args to init > - optionally: include an invoker flag to optionally remove action-configured > params from being sent to run (these would only be available by way of init > setting env vars, or exposing some other object) > - runtimes can be updated incrementally to support this flagged invoker > behavior > > I understand the point on amount of work, I'm just not wanting to sacrifice > an awkward behavior (different treatment for params in upper case) for sake > of time. There are some accumulating issues around init and run so I'm not > sure it is worth making the problem worse before addressing the other issues. > I think its ok to change init to receive these params as a minimal change, > but obviously without additional changes either at invoker or in OW's > convention for designing functions (main sig, how to access init vars, > context) there will be some incremental pain as well. > > Still curious what others think. > > > On 6/26/19, 11:27 AM, "Rodric Rabbah" wrote: > >> Sorry, it still seems controversial to me, not sure how others feel? > >That's why we discuss on the dev list :D Thanks for the feedback so far. > >> can you confirm this is decided based on the case of the parameter name? > >Indeed, we need some rule to then partition the parameter list. Using the >convention that the env var starts with a capital letter is one. Other >conventions are plausible. > >> adding a '-e' flag that specifically does "set these environment >variables" > >Sure - but this increases the complexity of implementation significantly >for not a lot of gain. To add a -e, we'd need to modify the schema for >actions. For example, we could add annotation for each parameter name to be >treated as an environment variable using the existing annotations, and use >these annotations as the criteria. We could create a new field in the >actions object to hold the parameters (a schema change). We could annotate >each parameter (also a schema change). > >Since a developer already controls the names of their parameters today, >they have complete control over this partitioning. > >If we're open to schema changes, then we can explore a cleaner >implementation but an incremental approach that at least makes the feature >available incrementally would also make sense since making a schema change >is a lot more invasive, coupled with a few changes needed at the invoker >level plus all the runtimes. > >-r > > > > -- Development thoughts at http://akrabat.com Daily Jotter for macOS at http://dailyjotter.com
Re: exporting activation arguments to the environment
I understand that this is orthogonal on what Rodric is proposing but for the topic of environment variables Reserved 2 annotations to be private like “__OW_CONFIGS” and “__OW_SECRETS” No schema change This variables would be dictionaries/jsonobjects The information will end up as environment variables at runtime. Then provide an user experience thru UI or CLI to set both Example of CLI: wsk action update myaction myaction.js -c LOGLEVEV DEBUG -s PASSWORD supersecret Both LOGLEVEL and PASSWORD will be set as environment variables The idea to brake them into two sets give the opportunity to the operator and the management platform to treat secrets a bit different if desired. - Carlos Santana @csantanapr > On Jun 27, 2019, at 6:00 AM, Dominic Kim wrote: > > I am inclined to be more explicit. > Whenever users forget about the difference between uppercase and lowercase > parameters, the feature may not work as they expected. > So I am inclined to Tyson's opinion to explicitly add another flag such as > "-e". > > If it could be overhead to change the schema, how about relying on > annotation? > I think it makes sense to set environment variables at action creation time. > We can introduce a new annotation to set environment variables, or apply > some rules against the existing flag "-a" as Rabbah suggested. > > Best regards > Dominic > > > > > > > 2019년 6월 27일 (목) 오전 8:52, Rodric Rabbah 님이 작성: > >> The use of env vars wouldn't be an issue with intra-container concurrency >> if they're immutable, right? The more specific issue that arises today >> which I think is your primary concern, are the __OW_ system provided >> environment variables which mutate with each invocation of main. Is that >> right? >> >> If so, then I think there are only two issues: >> 1. do we introduce a context object (did you see my other dev list mail?) >> 2. logging >> >> i really think my issue is orthogonal to these concerns - it's a >> convenience feature for the developer. An action can already export >> environment variables today from main. >> >> -r >> >> On Wed, Jun 26, 2019 at 8:44 PM Tyson Norris >> wrote: >> >>> Sorry, what I meant was: accumulating issues around the "main" function, >>> which are: >>> - context >>> - use of env vars >>> - your issue: separating user provided params from developer-provided >>> params >>> - completely separate, but worth noting: logging (and env vars) in the >>> face of concurrency >>> >>> On 6/26/19, 5:29 PM, "Rodric Rabbah" wrote: >>> There are some accumulating issues around init and run. >>> >>>Which issues are these? >>> >>>-r >>> >>> >>> >>
Re: exporting activation arguments to the environment
I am inclined to be more explicit. Whenever users forget about the difference between uppercase and lowercase parameters, the feature may not work as they expected. So I am inclined to Tyson's opinion to explicitly add another flag such as "-e". If it could be overhead to change the schema, how about relying on annotation? I think it makes sense to set environment variables at action creation time. We can introduce a new annotation to set environment variables, or apply some rules against the existing flag "-a" as Rabbah suggested. Best regards Dominic 2019년 6월 27일 (목) 오전 8:52, Rodric Rabbah 님이 작성: > The use of env vars wouldn't be an issue with intra-container concurrency > if they're immutable, right? The more specific issue that arises today > which I think is your primary concern, are the __OW_ system provided > environment variables which mutate with each invocation of main. Is that > right? > > If so, then I think there are only two issues: > 1. do we introduce a context object (did you see my other dev list mail?) > 2. logging > > i really think my issue is orthogonal to these concerns - it's a > convenience feature for the developer. An action can already export > environment variables today from main. > > -r > > On Wed, Jun 26, 2019 at 8:44 PM Tyson Norris > wrote: > > > Sorry, what I meant was: accumulating issues around the "main" function, > > which are: > > - context > > - use of env vars > > - your issue: separating user provided params from developer-provided > > params > > - completely separate, but worth noting: logging (and env vars) in the > > face of concurrency > > > > On 6/26/19, 5:29 PM, "Rodric Rabbah" wrote: > > > > > There are some accumulating issues around init and run. > > > > Which issues are these? > > > > -r > > > > > > >
Re: exporting activation arguments to the environment
The use of env vars wouldn't be an issue with intra-container concurrency if they're immutable, right? The more specific issue that arises today which I think is your primary concern, are the __OW_ system provided environment variables which mutate with each invocation of main. Is that right? If so, then I think there are only two issues: 1. do we introduce a context object (did you see my other dev list mail?) 2. logging i really think my issue is orthogonal to these concerns - it's a convenience feature for the developer. An action can already export environment variables today from main. -r On Wed, Jun 26, 2019 at 8:44 PM Tyson Norris wrote: > Sorry, what I meant was: accumulating issues around the "main" function, > which are: > - context > - use of env vars > - your issue: separating user provided params from developer-provided > params > - completely separate, but worth noting: logging (and env vars) in the > face of concurrency > > On 6/26/19, 5:29 PM, "Rodric Rabbah" wrote: > > > There are some accumulating issues around init and run. > > Which issues are these? > > -r > > >
Re: exporting activation arguments to the environment
Sorry, what I meant was: accumulating issues around the "main" function, which are: - context - use of env vars - your issue: separating user provided params from developer-provided params - completely separate, but worth noting: logging (and env vars) in the face of concurrency On 6/26/19, 5:29 PM, "Rodric Rabbah" wrote: > There are some accumulating issues around init and run. Which issues are these? -r
Re: exporting activation arguments to the environment
> There are some accumulating issues around init and run. Which issues are these? -r
Re: exporting activation arguments to the environment
For the incremental change I would suggest: - include the action-configured params as args to init - optionally: include an invoker flag to optionally remove action-configured params from being sent to run (these would only be available by way of init setting env vars, or exposing some other object) - runtimes can be updated incrementally to support this flagged invoker behavior I understand the point on amount of work, I'm just not wanting to sacrifice an awkward behavior (different treatment for params in upper case) for sake of time. There are some accumulating issues around init and run so I'm not sure it is worth making the problem worse before addressing the other issues. I think its ok to change init to receive these params as a minimal change, but obviously without additional changes either at invoker or in OW's convention for designing functions (main sig, how to access init vars, context) there will be some incremental pain as well. Still curious what others think. On 6/26/19, 11:27 AM, "Rodric Rabbah" wrote: > Sorry, it still seems controversial to me, not sure how others feel? That's why we discuss on the dev list :D Thanks for the feedback so far. > can you confirm this is decided based on the case of the parameter name? Indeed, we need some rule to then partition the parameter list. Using the convention that the env var starts with a capital letter is one. Other conventions are plausible. > adding a '-e' flag that specifically does "set these environment variables" Sure - but this increases the complexity of implementation significantly for not a lot of gain. To add a -e, we'd need to modify the schema for actions. For example, we could add annotation for each parameter name to be treated as an environment variable using the existing annotations, and use these annotations as the criteria. We could create a new field in the actions object to hold the parameters (a schema change). We could annotate each parameter (also a schema change). Since a developer already controls the names of their parameters today, they have complete control over this partitioning. If we're open to schema changes, then we can explore a cleaner implementation but an incremental approach that at least makes the feature available incrementally would also make sense since making a schema change is a lot more invasive, coupled with a few changes needed at the invoker level plus all the runtimes. -r
Re: exporting activation arguments to the environment
By the way, an existential question is whether we should facilitate environment variables at all for actions (a user can still do whatever they like). To Rob's point about separating parameters, this could be achieved through environment variables, or context object (to tie the two dev threads together). -r
Re: exporting activation arguments to the environment
> Sorry, it still seems controversial to me, not sure how others feel?
That's why we discuss on the dev list :D Thanks for the feedback so far.
> can you confirm this is decided based on the case of the parameter name?
Indeed, we need some rule to then partition the parameter list. Using the
convention that the env var starts with a capital letter is one. Other
conventions are plausible.
> adding a '-e' flag that specifically does "set these environment
variables"
Sure - but this increases the complexity of implementation significantly
for not a lot of gain. To add a -e, we'd need to modify the schema for
actions. For example, we could add annotation for each parameter name to be
treated as an environment variable using the existing annotations, and use
these annotations as the criteria. We could create a new field in the
actions object to hold the parameters (a schema change). We could annotate
each parameter (also a schema change).
Since a developer already controls the names of their parameters today,
they have complete control over this partitioning.
If we're open to schema changes, then we can explore a cleaner
implementation but an incremental approach that at least makes the feature
available incrementally would also make sense since making a schema change
is a lot more invasive, coupled with a few changes needed at the invoker
level plus all the runtimes.
-r
On Wed, Jun 26, 2019 at 2:07 PM Tyson Norris
wrote:
> Sorry, it still seems controversial to me, not sure how others feel?
>
> To be clear, when you added "-a partition-arguments true", the result is
> 2 things:
> 1. some of the -p args are now treated different than others - can you
> confirm this is decided based on the case of the parameter name?
> 2. init receives these params (which sounds good to me).
>
> Regardless of opting in to this behavior, having action-configured
> parameters referenced differently based on the name of the param seems
> bad. I understand there are some useful conventions defining these as env
> vars, but my point is that this doesn't seem at all like an explicit
> choice. I think an explicit choice would be more like adding a '-e' flag
> that specifically does "set these environment variables", instead of
> overloading the '-p' flag with a convention based on the name of the
> variable.
>
> Thanks
> Tyson
>
>
> On 6/26/19, 10:43 AM, "Rodric Rabbah" wrote:
>
> Maybe this got missed, but here's how I conceived of this. I'll use
> wsk CLI
> commands, I think it makes it obvious.
>
> wsk action create myAction code.js -p MY_ENV true -p some_param false
> -a
> partition-arguments true
>
> The annotation (partition-arguments) makes it explicit for the
> developer to
> control whether "main" receives the arguments as they do today, which
> is
> this object
> { MY_ENV: true, some_param: false}, or when the annotation is true, {
> some_param: false} and process.env.MY_ENV is set to true.
>
> I don't think there's anything confusing about this in that the
> developer
> has decided what variables to export to the environment, and is making
> an
> explicit choice.
>
> Environment variables on a number of platforms are restricted to those
> at
> consist of words that start with capital letter (AWS, Netlify as two
> prime
> examples).
>
> The alternative, today, requires a function to export any variables
> from
> "main" to the environment. So it would explicitly export MY_ENV to the
> environment. The change we're discussing frees the programmer from
> having
> to do that.
>
> The change to the runtime proxies would be 1. accept an additional
> value on
> /init and export all the properties it contains to the environment.
>
> Before I address the POST invoke issue, I'd like to make sure my
> explanation is clearer and if this is still controversial.
>
> -r
>
>
> On Wed, Jun 26, 2019 at 1:21 PM Tyson Norris >
> wrote:
>
> > Are you saying one is exported to environment, during init, based on
> > parameter name being UPPER case? Forgetting use of env vars for a
> minute,
> > this seems confusing to treat parameters different based on names. I
> would
> > rather see either a) all action-configured params sent to init only,
> and
> > never to run or b) all action-configured params sent to run as
> context
> > object.
> >
> > What the runtime does at init (use env vars or not) can be different
> per
> > runtime, but in the action-configured parameter case I don't see any
> > problem with setting env vars, except that there seems to be a
> convention
> > in some cases that allows invoking clients to "override" these
> values using
> > POST parameters at invocation time. This also seems confusing but
> could
> > also be enforced differently by various runtimes, although ideally I
> would
> > rather see the convention change to: action-configured parameters are
Re: exporting activation arguments to the environment
Sorry, it still seems controversial to me, not sure how others feel?
To be clear, when you added "-a partition-arguments true", the result is 2
things:
1. some of the -p args are now treated different than others - can you confirm
this is decided based on the case of the parameter name?
2. init receives these params (which sounds good to me).
Regardless of opting in to this behavior, having action-configured parameters
referenced differently based on the name of the param seems bad. I understand
there are some useful conventions defining these as env vars, but my point is
that this doesn't seem at all like an explicit choice. I think an explicit
choice would be more like adding a '-e' flag that specifically does "set these
environment variables", instead of overloading the '-p' flag with a convention
based on the name of the variable.
Thanks
Tyson
On 6/26/19, 10:43 AM, "Rodric Rabbah" wrote:
Maybe this got missed, but here's how I conceived of this. I'll use wsk CLI
commands, I think it makes it obvious.
wsk action create myAction code.js -p MY_ENV true -p some_param false -a
partition-arguments true
The annotation (partition-arguments) makes it explicit for the developer to
control whether "main" receives the arguments as they do today, which is
this object
{ MY_ENV: true, some_param: false}, or when the annotation is true, {
some_param: false} and process.env.MY_ENV is set to true.
I don't think there's anything confusing about this in that the developer
has decided what variables to export to the environment, and is making an
explicit choice.
Environment variables on a number of platforms are restricted to those at
consist of words that start with capital letter (AWS, Netlify as two prime
examples).
The alternative, today, requires a function to export any variables from
"main" to the environment. So it would explicitly export MY_ENV to the
environment. The change we're discussing frees the programmer from having
to do that.
The change to the runtime proxies would be 1. accept an additional value on
/init and export all the properties it contains to the environment.
Before I address the POST invoke issue, I'd like to make sure my
explanation is clearer and if this is still controversial.
-r
On Wed, Jun 26, 2019 at 1:21 PM Tyson Norris
wrote:
> Are you saying one is exported to environment, during init, based on
> parameter name being UPPER case? Forgetting use of env vars for a minute,
> this seems confusing to treat parameters different based on names. I would
> rather see either a) all action-configured params sent to init only, and
> never to run or b) all action-configured params sent to run as context
> object.
>
> What the runtime does at init (use env vars or not) can be different per
> runtime, but in the action-configured parameter case I don't see any
> problem with setting env vars, except that there seems to be a convention
> in some cases that allows invoking clients to "override" these values
using
> POST parameters at invocation time. This also seems confusing but could
> also be enforced differently by various runtimes, although ideally I would
> rather see the convention change to: action-configured parameters are
> always sent to init, and always visible to run, regardless of what client
> sends as execution parameters.
>
> Thanks
> Tyson
>
>
> On 6/25/19, 3:32 PM, "Rodric Rabbah" wrote:
>
> Context and Knative I view as orthogonal.
>
> That is, for the context object, it is another way of encapsulating
> arguments. It doesn’t export variable to the process environment.
>
> You can provide an action with both environment variables, arguments
> to main, and a context object. They are orthogonal.
>
> For the context object, the distinction that was necessary from
> previous discussions was related to separating intra container concurrent
> executions. If the system-provided context is exported to the environment
> as it today the values clobber each other. For this, the context object
> would make sense.
>
> I’m simply talking about two parameters wsk ... “-p a A” and “-p B b”
> say where one becomes exported to the environment as B=b and the other is
> passed to the action as ({a:A}).
>
> I’m going to set the knative discussion aside because I think it’s a
> distraction. With knative you can bind environment variables to the
> container. As you would with any other container.
>
> I think it’s too simplistic to say knative has a single endpoint.
> After all there are readiness probes and possible pre/post start hooks
that
> operators may have to deal with. Init can be viewed as the readiness
Re: exporting activation arguments to the environment
Maybe this got missed, but here's how I conceived of this. I'll use wsk CLI
commands, I think it makes it obvious.
wsk action create myAction code.js -p MY_ENV true -p some_param false -a
partition-arguments true
The annotation (partition-arguments) makes it explicit for the developer to
control whether "main" receives the arguments as they do today, which is
this object
{ MY_ENV: true, some_param: false}, or when the annotation is true, {
some_param: false} and process.env.MY_ENV is set to true.
I don't think there's anything confusing about this in that the developer
has decided what variables to export to the environment, and is making an
explicit choice.
Environment variables on a number of platforms are restricted to those at
consist of words that start with capital letter (AWS, Netlify as two prime
examples).
The alternative, today, requires a function to export any variables from
"main" to the environment. So it would explicitly export MY_ENV to the
environment. The change we're discussing frees the programmer from having
to do that.
The change to the runtime proxies would be 1. accept an additional value on
/init and export all the properties it contains to the environment.
Before I address the POST invoke issue, I'd like to make sure my
explanation is clearer and if this is still controversial.
-r
On Wed, Jun 26, 2019 at 1:21 PM Tyson Norris
wrote:
> Are you saying one is exported to environment, during init, based on
> parameter name being UPPER case? Forgetting use of env vars for a minute,
> this seems confusing to treat parameters different based on names. I would
> rather see either a) all action-configured params sent to init only, and
> never to run or b) all action-configured params sent to run as context
> object.
>
> What the runtime does at init (use env vars or not) can be different per
> runtime, but in the action-configured parameter case I don't see any
> problem with setting env vars, except that there seems to be a convention
> in some cases that allows invoking clients to "override" these values using
> POST parameters at invocation time. This also seems confusing but could
> also be enforced differently by various runtimes, although ideally I would
> rather see the convention change to: action-configured parameters are
> always sent to init, and always visible to run, regardless of what client
> sends as execution parameters.
>
> Thanks
> Tyson
>
>
> On 6/25/19, 3:32 PM, "Rodric Rabbah" wrote:
>
> Context and Knative I view as orthogonal.
>
> That is, for the context object, it is another way of encapsulating
> arguments. It doesn’t export variable to the process environment.
>
> You can provide an action with both environment variables, arguments
> to main, and a context object. They are orthogonal.
>
> For the context object, the distinction that was necessary from
> previous discussions was related to separating intra container concurrent
> executions. If the system-provided context is exported to the environment
> as it today the values clobber each other. For this, the context object
> would make sense.
>
> I’m simply talking about two parameters wsk ... “-p a A” and “-p B b”
> say where one becomes exported to the environment as B=b and the other is
> passed to the action as ({a:A}).
>
> I’m going to set the knative discussion aside because I think it’s a
> distraction. With knative you can bind environment variables to the
> container. As you would with any other container.
>
> I think it’s too simplistic to say knative has a single endpoint.
> After all there are readiness probes and possible pre/post start hooks that
> operators may have to deal with. Init can be viewed as the readiness probe.
>
> Fundamentally I believe the actor model is much better aligned with
> the reactive programming model for functions so this will tend toward a
> completely different discussion in my view.
>
> The reason my proposal sets the environment variables at init time is
> that’s how env vars work; they exist before you start you process. While
> they don’t need to be immutable, it makes sense to test them as such.
>
> For webaction parameters that one would export to an environment, they
> are already immutable and cannot be overridden. So really you would not use
> them for anything that varies per activation.
>
> The view here is that you can export global (immutable) variables to
> the action. This makes it easier to take existing code and containers which
> might use env vars and use them almost off the shelf.
>
> -r
>
> > On Jun 25, 2019, at 6:07 PM, Tyson Norris
> wrote:
> >
> > I had to read this several times, but have some suggestions. I think
> when you say "action's arguments", you mean action-configured params, e.g.
> `wsk action create --param p1 v1`?
> >
> > My preferences would be:
> > - we should split off "run" args into context and params - this is
> the convention change for redef
Re: exporting activation arguments to the environment
Are you saying one is exported to environment, during init, based on parameter
name being UPPER case? Forgetting use of env vars for a minute, this seems
confusing to treat parameters different based on names. I would rather see
either a) all action-configured params sent to init only, and never to run or
b) all action-configured params sent to run as context object.
What the runtime does at init (use env vars or not) can be different per
runtime, but in the action-configured parameter case I don't see any problem
with setting env vars, except that there seems to be a convention in some cases
that allows invoking clients to "override" these values using POST parameters
at invocation time. This also seems confusing but could also be enforced
differently by various runtimes, although ideally I would rather see the
convention change to: action-configured parameters are always sent to init, and
always visible to run, regardless of what client sends as execution parameters.
Thanks
Tyson
On 6/25/19, 3:32 PM, "Rodric Rabbah" wrote:
Context and Knative I view as orthogonal.
That is, for the context object, it is another way of encapsulating
arguments. It doesn’t export variable to the process environment.
You can provide an action with both environment variables, arguments to
main, and a context object. They are orthogonal.
For the context object, the distinction that was necessary from previous
discussions was related to separating intra container concurrent executions. If
the system-provided context is exported to the environment as it today the
values clobber each other. For this, the context object would make sense.
I’m simply talking about two parameters wsk ... “-p a A” and “-p B b” say
where one becomes exported to the environment as B=b and the other is passed to
the action as ({a:A}).
I’m going to set the knative discussion aside because I think it’s a
distraction. With knative you can bind environment variables to the container.
As you would with any other container.
I think it’s too simplistic to say knative has a single endpoint. After all
there are readiness probes and possible pre/post start hooks that operators may
have to deal with. Init can be viewed as the readiness probe.
Fundamentally I believe the actor model is much better aligned with the
reactive programming model for functions so this will tend toward a completely
different discussion in my view.
The reason my proposal sets the environment variables at init time is
that’s how env vars work; they exist before you start you process. While they
don’t need to be immutable, it makes sense to test them as such.
For webaction parameters that one would export to an environment, they are
already immutable and cannot be overridden. So really you would not use them
for anything that varies per activation.
The view here is that you can export global (immutable) variables to the
action. This makes it easier to take existing code and containers which might
use env vars and use them almost off the shelf.
-r
> On Jun 25, 2019, at 6:07 PM, Tyson Norris
wrote:
>
> I had to read this several times, but have some suggestions. I think when
you say "action's arguments", you mean action-configured params, e.g. `wsk
action create --param p1 v1`?
>
> My preferences would be:
> - we should split off "run" args into context and params - this is the
convention change for redefining main(args) as main(context, args) we have
discussed in the past.
> - I support either having init receive action-configured params
> - activation args that are possibly overridden should behave exactly as
specified args - is it important that action-configured args are actually
overridden, if the context and params are separated? (receive both values, and
logic must decide when to use which)
> - let's not use env variables for any arg that is variable per activation
- it is impossible if you support concurrency, and unneeded if we pass the
context to "run".
>
> Regarding Matt's suggestion to remove init - I like this idea, but I have
concerns compared to knative which might serve every function with a different
container, vs having some containers reused for multiple functions. In the case
where we init code into an already running container, it is useful to have the
init process separate from run, since otherwise each runtime will need to track
its own init state and queue requests during init etc. If I'm not getting the
whole picture with knative, please correct me.
>
>
> Thanks
> Tyson
>
> On 6/24/19, 8:43 AM, "Rodric Rabbah" wrote:
>
>In the current activation model, an action's arguments are always
provided
>to the action on "run", not "init".
>
>Should we consider partitioning the argument list into two
Re: exporting activation arguments to the environment
Hi,
I have always been in favour of separating the parameters that are set by the
developer/environment (context) from those that are input data for this
particular-activation. For me its about which ones we trust and which ones we
don't.
i.e. database credentials and the activation id are conceptually different from
POSTed data and this should be obvious in my code as it means fewer mistakes
and makes code-reviews that much easier.
Regards,
Rob
> On 25 Jun 2019, at 23:07, Tyson Norris wrote:
>
> I had to read this several times, but have some suggestions. I think when you
> say "action's arguments", you mean action-configured params, e.g. `wsk action
> create --param p1 v1`?
>
> My preferences would be:
> - we should split off "run" args into context and params - this is the
> convention change for redefining main(args) as main(context, args) we have
> discussed in the past.
> - I support either having init receive action-configured params
> - activation args that are possibly overridden should behave exactly as
> specified args - is it important that action-configured args are actually
> overridden, if the context and params are separated? (receive both values,
> and logic must decide when to use which)
> - let's not use env variables for any arg that is variable per activation -
> it is impossible if you support concurrency, and unneeded if we pass the
> context to "run".
>
> Regarding Matt's suggestion to remove init - I like this idea, but I have
> concerns compared to knative which might serve every function with a
> different container, vs having some containers reused for multiple functions.
> In the case where we init code into an already running container, it is
> useful to have the init process separate from run, since otherwise each
> runtime will need to track its own init state and queue requests during init
> etc. If I'm not getting the whole picture with knative, please correct me.
>
>
> Thanks
> Tyson
>
> On 6/24/19, 8:43 AM, "Rodric Rabbah" wrote:
>
>In the current activation model, an action's arguments are always provided
>to the action on "run", not "init".
>
>Should we consider partitioning the argument list into two sets, the first
>is exported as environment variables at "init" time, and the second become
>the action's argument at "run" time? A criteria for partitioning is that
>the environment variable starts with a capital letter, which is a common
>convention.
>
>For example, an action which is invoked with a JSON object
>
>{ "XYZ": true,
> "abc" : false }
>
>would receive {"abc": false} as its arguments and can read XYZ from the
>environment (as process.env.XYZ == "true" in Node.js).
>
>This change would:
>1. require a change in the invoker to pass arguments during initialization
>
>2. require a change in the runtime proxies to export the arguments to the
>environment at initialization time (additional work may be implied by 1b)
>
>3. an annotation on actions to opt into this partitioning for backward
>compatibility or to opt out. For example '-a env-partition-arguments true'
>partitions the arguments and actions without this annotation are not
>affected.
>
>Some obvious question:
>Q1a. should the invoker perform the partitioning or delegate it to the
>runtime? The advantage of the former is that the runtimes do not have to
>implement the filtering policy and do less work. I think it makes sense to
>do this invoker side for uniformity.
>
>Q1b. should the partitioning treat environment variables as immutable post
>init and ignore the partition on warm starts? This is an issue when a value
>is overridden during POST invoke only since for a webaction, you cannot
>override a value that's already defined (and updating a bound parameter on
>an action invalidates warm containers). I think env vars should be treated
>as immutable despite the issue with POST invoke.
>
>-r
>
>
--
Development thoughts at http://akrabat.com
Daily Jotter for macOS at http://dailyjotter.com
Re: exporting activation arguments to the environment
Context and Knative I view as orthogonal.
That is, for the context object, it is another way of encapsulating arguments.
It doesn’t export variable to the process environment.
You can provide an action with both environment variables, arguments to main,
and a context object. They are orthogonal.
For the context object, the distinction that was necessary from previous
discussions was related to separating intra container concurrent executions. If
the system-provided context is exported to the environment as it today the
values clobber each other. For this, the context object would make sense.
I’m simply talking about two parameters wsk ... “-p a A” and “-p B b” say
where one becomes exported to the environment as B=b and the other is passed to
the action as ({a:A}).
I’m going to set the knative discussion aside because I think it’s a
distraction. With knative you can bind environment variables to the container.
As you would with any other container.
I think it’s too simplistic to say knative has a single endpoint. After all
there are readiness probes and possible pre/post start hooks that operators may
have to deal with. Init can be viewed as the readiness probe.
Fundamentally I believe the actor model is much better aligned with the
reactive programming model for functions so this will tend toward a completely
different discussion in my view.
The reason my proposal sets the environment variables at init time is that’s
how env vars work; they exist before you start you process. While they don’t
need to be immutable, it makes sense to test them as such.
For webaction parameters that one would export to an environment, they are
already immutable and cannot be overridden. So really you would not use them
for anything that varies per activation.
The view here is that you can export global (immutable) variables to the
action. This makes it easier to take existing code and containers which might
use env vars and use them almost off the shelf.
-r
> On Jun 25, 2019, at 6:07 PM, Tyson Norris wrote:
>
> I had to read this several times, but have some suggestions. I think when you
> say "action's arguments", you mean action-configured params, e.g. `wsk action
> create --param p1 v1`?
>
> My preferences would be:
> - we should split off "run" args into context and params - this is the
> convention change for redefining main(args) as main(context, args) we have
> discussed in the past.
> - I support either having init receive action-configured params
> - activation args that are possibly overridden should behave exactly as
> specified args - is it important that action-configured args are actually
> overridden, if the context and params are separated? (receive both values,
> and logic must decide when to use which)
> - let's not use env variables for any arg that is variable per activation -
> it is impossible if you support concurrency, and unneeded if we pass the
> context to "run".
>
> Regarding Matt's suggestion to remove init - I like this idea, but I have
> concerns compared to knative which might serve every function with a
> different container, vs having some containers reused for multiple functions.
> In the case where we init code into an already running container, it is
> useful to have the init process separate from run, since otherwise each
> runtime will need to track its own init state and queue requests during init
> etc. If I'm not getting the whole picture with knative, please correct me.
>
>
> Thanks
> Tyson
>
> On 6/24/19, 8:43 AM, "Rodric Rabbah" wrote:
>
>In the current activation model, an action's arguments are always provided
>to the action on "run", not "init".
>
>Should we consider partitioning the argument list into two sets, the first
>is exported as environment variables at "init" time, and the second become
>the action's argument at "run" time? A criteria for partitioning is that
>the environment variable starts with a capital letter, which is a common
>convention.
>
>For example, an action which is invoked with a JSON object
>
>{ "XYZ": true,
> "abc" : false }
>
>would receive {"abc": false} as its arguments and can read XYZ from the
>environment (as process.env.XYZ == "true" in Node.js).
>
>This change would:
>1. require a change in the invoker to pass arguments during initialization
>
>2. require a change in the runtime proxies to export the arguments to the
>environment at initialization time (additional work may be implied by 1b)
>
>3. an annotation on actions to opt into this partitioning for backward
>compatibility or to opt out. For example '-a env-partition-arguments true'
>partitions the arguments and actions without this annotation are not
>affected.
>
>Some obvious question:
>Q1a. should the invoker perform the partitioning or delegate it to the
>runtime? The advantage of the former is that the runtimes do not
Re: exporting activation arguments to the environment
I had to read this several times, but have some suggestions. I think when you
say "action's arguments", you mean action-configured params, e.g. `wsk action
create --param p1 v1`?
My preferences would be:
- we should split off "run" args into context and params - this is the
convention change for redefining main(args) as main(context, args) we have
discussed in the past.
- I support either having init receive action-configured params
- activation args that are possibly overridden should behave exactly as
specified args - is it important that action-configured args are actually
overridden, if the context and params are separated? (receive both values, and
logic must decide when to use which)
- let's not use env variables for any arg that is variable per activation - it
is impossible if you support concurrency, and unneeded if we pass the context
to "run".
Regarding Matt's suggestion to remove init - I like this idea, but I have
concerns compared to knative which might serve every function with a different
container, vs having some containers reused for multiple functions. In the case
where we init code into an already running container, it is useful to have the
init process separate from run, since otherwise each runtime will need to track
its own init state and queue requests during init etc. If I'm not getting the
whole picture with knative, please correct me.
Thanks
Tyson
On 6/24/19, 8:43 AM, "Rodric Rabbah" wrote:
In the current activation model, an action's arguments are always provided
to the action on "run", not "init".
Should we consider partitioning the argument list into two sets, the first
is exported as environment variables at "init" time, and the second become
the action's argument at "run" time? A criteria for partitioning is that
the environment variable starts with a capital letter, which is a common
convention.
For example, an action which is invoked with a JSON object
{ "XYZ": true,
"abc" : false }
would receive {"abc": false} as its arguments and can read XYZ from the
environment (as process.env.XYZ == "true" in Node.js).
This change would:
1. require a change in the invoker to pass arguments during initialization
2. require a change in the runtime proxies to export the arguments to the
environment at initialization time (additional work may be implied by 1b)
3. an annotation on actions to opt into this partitioning for backward
compatibility or to opt out. For example '-a env-partition-arguments true'
partitions the arguments and actions without this annotation are not
affected.
Some obvious question:
Q1a. should the invoker perform the partitioning or delegate it to the
runtime? The advantage of the former is that the runtimes do not have to
implement the filtering policy and do less work. I think it makes sense to
do this invoker side for uniformity.
Q1b. should the partitioning treat environment variables as immutable post
init and ignore the partition on warm starts? This is an issue when a value
is overridden during POST invoke only since for a webaction, you cannot
override a value that's already defined (and updating a bound parameter on
an action invalidates warm containers). I think env vars should be treated
as immutable despite the issue with POST invoke.
-r
Re: exporting activation arguments to the environment
Hi Rodric, Have many thoughts on this having just experienced them all when mapping our OW "runtime contract" to Knative... but first would ask a couple of things based on your historic knowledge... M1) Do you have a specific use case which highlights the issue (i.e., caused you to think on this at this time)? M2) what was the intent of the separation of between init and run (esp. as it seems other impl.s of Serverless like Knative are moving to single exported entrypoint). Will there ever be a "re-init" to reuse a runtime (framework). Will we explore a V8 Isolates approach? 3M) It seems knative thinking is 12-factor app... everything in ENV of container whereas, as a functions (as a workload) developer would hopefully attempt to perform their task without any environmental (OS) awareness ans strictly remain within the function scope. Should we not seek to re-enforce the Serverless differentiation? or give into the notion that functions are just "baked into" Containers. After our knative work, I was actually thinking that init and run distinction should go away as init serves little purpose... that we should endeavor to have a unified container proxy that was a single point enforcement/data massaging for all runtimes (seemingly moving in this direction naturally anyways). Then use the container proxy to explore new use cases uniformly (e.g., other protocols, etc.).
exporting activation arguments to the environment
In the current activation model, an action's arguments are always provided
to the action on "run", not "init".
Should we consider partitioning the argument list into two sets, the first
is exported as environment variables at "init" time, and the second become
the action's argument at "run" time? A criteria for partitioning is that
the environment variable starts with a capital letter, which is a common
convention.
For example, an action which is invoked with a JSON object
{ "XYZ": true,
"abc" : false }
would receive {"abc": false} as its arguments and can read XYZ from the
environment (as process.env.XYZ == "true" in Node.js).
This change would:
1. require a change in the invoker to pass arguments during initialization
2. require a change in the runtime proxies to export the arguments to the
environment at initialization time (additional work may be implied by 1b)
3. an annotation on actions to opt into this partitioning for backward
compatibility or to opt out. For example '-a env-partition-arguments true'
partitions the arguments and actions without this annotation are not
affected.
Some obvious question:
Q1a. should the invoker perform the partitioning or delegate it to the
runtime? The advantage of the former is that the runtimes do not have to
implement the filtering policy and do less work. I think it makes sense to
do this invoker side for uniformity.
Q1b. should the partitioning treat environment variables as immutable post
init and ignore the partition on warm starts? This is an issue when a value
is overridden during POST invoke only since for a webaction, you cannot
override a value that's already defined (and updating a bound parameter on
an action invalidates warm containers). I think env vars should be treated
as immutable despite the issue with POST invoke.
-r
