Re: [openstack-dev] [TaskFlow] Cross-run persistence
> On Aug 12, 2015, at 12:49 PM, Joshua Harlow wrote: > > possibly submit some kind of oslo-spec or > blueprint for 'selective persistence' (or other better named thing); if > u want to work on said code, that's even better :) Cheers, will do.. Once I get legal approval from work to contribute. --- GPG Fingerprint: 9530 B4AF 551B F3CD A45C 476C D4E5 662D DB97 69E3 signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TaskFlow] Cross-run persistence
On Wed, 12 Aug 2015 10:31:03 -0700
Demian Brecht wrote:
>
> > On Aug 10, 2015, at 1:18 PM, Joshua Harlow
> > wrote:
> >
> > Is that what u are looking for? (or possibly something else?),
>
> Hi Josh,
>
> Thanks for the reply. I haven’t had time to dig into it much further,
> but I’m not sure that’s what I’m looking for (unless I’m missing
> something in initial configuration. What I have tried so far is
> something to the effect of:
>
> store = {‘foo’: ‘bar’}
> class MyAuthTask(task.Task):
> default_provides = ‘auth_token’
>
> def execute(self):
> return ‘mytoken'
>
> engine = engines.load(flow, …, store=store)
> engine.run()
> print(store)
>
> What I’m trying to get at is to selectively persist the token. In
> this case, after engine execution, values injected to the store are
> no longer there. If I understand how the engine works (which I may
> not, I’ve only been looking into this /very/ lightly over the last
> few days), as tasks are executed, what they provide is injected into
> the store. Unless you specifically tell the system to not
> purge /anything/ from the store, all will be lost once the engine is
> complete. I also don’t want to /all/ data to persist, but only select
> items (i.e. tokens).
Hmmm, I'll be out on vacation for a couple weeks, so feel free to jump
on IRC and ask there, but the summary u have is mostly correct, but
what u have to do that I don't see in the above is specify which
persistence backend the engine should be using, if none is selected
then all data saved goes into memory (and therefore is lost when the
program is done).
Once u get that working things will be persisted, although selective
persistence isn't currently possible, but could be done in a somewhat
easy manner (as long as we are clear on what i means and/or implies),
Once u get the persistence working,
http://docs.openstack.org/developer/taskflow/persistence.html should
help here then feel free to possibly submit some kind of oslo-spec or
blueprint for 'selective persistence' (or other better named thing); if
u want to work on said code, that's even better :)
Spec template @
https://github.com/openstack/oslo-specs/tree/master/specs
Irc folks usually @ #openstack-state-management and/or #openstack-oslo
(should be folks there that can help while i'm out...)
-Josh
>
> Am I misunderstanding something here?
>
> Thanks again,
> Demian
>
> ---
> GPG Fingerprint: 9530 B4AF 551B F3CD A45C 476C D4E5 662D DB97 69E3
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TaskFlow] Cross-run persistence
> On Aug 10, 2015, at 1:18 PM, Joshua Harlow wrote:
>
> Is that what u are looking for? (or possibly something else?),
Hi Josh,
Thanks for the reply. I haven’t had time to dig into it much further, but I’m
not sure that’s what I’m looking for (unless I’m missing something in initial
configuration. What I have tried so far is something to the effect of:
store = {‘foo’: ‘bar’}
class MyAuthTask(task.Task):
default_provides = ‘auth_token’
def execute(self):
return ‘mytoken'
engine = engines.load(flow, …, store=store)
engine.run()
print(store)
What I’m trying to get at is to selectively persist the token. In this case,
after engine execution, values injected to the store are no longer there. If I
understand how the engine works (which I may not, I’ve only been looking into
this /very/ lightly over the last few days), as tasks are executed, what they
provide is injected into the store. Unless you specifically tell the system to
not purge /anything/ from the store, all will be lost once the engine is
complete. I also don’t want to /all/ data to persist, but only select items
(i.e. tokens).
Am I misunderstanding something here?
Thanks again,
Demian
---
GPG Fingerprint: 9530 B4AF 551B F3CD A45C 476C D4E5 662D DB97 69E3
signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TaskFlow] Cross-run persistence
Hmmm, I'm pretty sure what u want exists, For example: https://github.com/openstack/taskflow/blob/master/taskflow/examples/persistence_example.py That uses sqlite (or a file) to store information, U just have to decide what backend is best for u, Is that what u are looking for? (or possibly something else?), -Josh Demian Brecht wrote: Hi all, As I understand it, the persistence and related backends are intended to store atom and flow state. This will only persist until the end of an entirely successful run (unless explicitly configured to not purge for auditing and other such reasons). My question is: Is there a way to configure storage to persist data across runs? My specific use case: I’ve defined an authentication task. The auth server allows for ticket-based auth, which means the user (optionally) wouldn’t have to log in on every run if storage could be configured to persist the given token across runs. Did I miss something in the docs about this use case or should I be looking at something like oslo.cache to solve this problem? Thanks, Demian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TaskFlow] Cross-run persistence
Hi all, As I understand it, the persistence and related backends are intended to store atom and flow state. This will only persist until the end of an entirely successful run (unless explicitly configured to not purge for auditing and other such reasons). My question is: Is there a way to configure storage to persist data across runs? My specific use case: I’ve defined an authentication task. The auth server allows for ticket-based auth, which means the user (optionally) wouldn’t have to log in on every run if storage could be configured to persist the given token across runs. Did I miss something in the docs about this use case or should I be looking at something like oslo.cache to solve this problem? Thanks, Demian signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
