Since the uploading of JS is an interesting concept, let me just share a little about another service at Y! that has done this.
http://developer.yahoo.com/yql/guide/yql-execute-chapter.html YQL (although closed-source) has exposed this ability, so its entirely possible to do (although it likely won't be running in python). YQL itself is slightly similar to a DSL also (its DSL is similar to SQL). The nice thing about using something like JS (to continue along this idea, even if nobody wants to actually implement this) is that things like rhino<http://en.wikipedia.org/wiki/Rhino_%28JavaScript_engine%29> do provide execution limits (at the instruction level). Of course this would potentially bring in java (I am guessing node.js can do something similar as rhino). Anyways… To further this lets continue working on https://etherpad.openstack.org/p/taskflow-mistral and see if we can align somehow (I hope it's not to late to do this, seeing that there appears to be a lot of resistance from the mistral community to change). But I agree with clint, and hope that we can have a healthy collaboration as a community instead of being in competing silos (which is not healthy). -Josh From: Clint Byrum <[email protected]<mailto:[email protected]>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <[email protected]<mailto:[email protected]>> Date: Monday, March 17, 2014 at 9:41 AM To: openstack-dev <[email protected]<mailto:[email protected]>> Subject: Re: [openstack-dev] [Mistral] Actions design BP Excerpts from Renat Akhmerov's message of 2014-03-17 07:35:02 -0700: On 16 Mar 2014, at 19:05, Clint Byrum <[email protected]<mailto:[email protected]>> wrote: > From my perspective, as somebody who is considering Mistral for some > important things, the fact that the taskflow people are not aligned with > the Mistral people tells me that Mistral is not what I thought it was: > taskflow for direct user consumption. Yes, it was just an initial idea we were trying to pursue. As we moved forward we understood it was too limited and had a lot of pitfalls. The reason is the key difference between library and service. Making something a service flips everything upside down in terms of requirements to this new system. The logical questions/implications are: If that’s a service why should its REST API be Python oriented? Bindings - yes (it gives a certain convenience) but that’s a different story.. A whole set of questions related to how we distribute Python-written tasks (especially in multi tenant environment, e.g. as a cloud hosted service): Dependencies Sandboxing Result serialisation etc. It becomes logical to be able to use it with non-python clients and external systems. Orientation to long-lived processes(workflows). Synchronous execution model no longer works well, instead de facto asynchronous event-based architecture fits better. I know it may sound too generic, it’s a whole different topic.. It becomes logical to provide more high-level capabilities rather than a library does (e.g. Horizon Dashboard. It would not be much of use if the service was based on Python). I assume Mistral is written in Python though, and so it should be using Taskflow for its own workflow. I understand though, that to run _user_ workflows you can't just expect them to upload python or always run python as Mistral wouldn't do much for them at that point. However, for the User<->Taskflow integration, I think Josh Harlow offered a really reasonable suggestion to that, which is instead of inventing a DSL (frankly, I'm pretty frustrated with the Heat DSL's we have, so I may be biased here), just embrace javascript or lua, which are designed to be embedded and executed in a less-than-trusted context. I think it would be a great story for users if Mistral worked like this: - Upload javascript expression of workflow, with external callouts for complicated things. - Run code that uses Mistral API to poll-for or subscribe-to notifications waiting for instructions from Mistral when it is supposed to run said external callouts, and feeds back data. I believe it’s not the full list. Despite of all this I’m not against of using TaskFlow in implementation, at least some pieces of it. My only call is: it should be really beneficial rather than bringing pain. So I’m ok if we keep aligning our capabilities, roadmaps, terminology and whatever else is needed. Btw, it would be really helpful if you could clarify what you meant “some important things”. Asking because I still feel like we could have much more input from potential users. Any information is appreciated. - Heat needs to move from being a single state machine (heat-engine owns all of the running tasks for a stack in one engine at a time) to a distributed state machine. Before we do that, we need to consider how Heat expresses workflow. If Mistral were a distributed workflow engine, it would make a lot of sense for Heat to make use of it for this purpose. - TripleO deploys a number of things that need distributed workflow. I think at this point the people involved with that are looking more at lower level tools like RAFT and Concoord. But once the distributed state machine is settled, there will be a need to express the distributed workflow. I'm disinclined to diverge from Taskflow, even though I am quite inclined to embrace API's. > So, while it may be annoying to have your "day to day project > activities" questioned, it is pretty core to the discussion considering > that this suggests several things that diverge from taskflow's core > model. Np. I still keep finding things/rules in OpenStack that I’m not really aware of or didn’t get used to yet. If that’s not how it should be done in OS then it’s fine. I'm sorry if the message was harsh, but I see this happening a lot. I don't think it is a rule. I think the principle here is to collaborate on things that should be aligned. If Taskflow doesn't do what you need it to do, then I suggest _fixing that_ rather than writing a private implementation. It will make life better for all users of taskflow and it will keep Mistral extremely simple, which will help with adoption by operators _and_ users. _______________________________________________ OpenStack-dev mailing list [email protected]<mailto:[email protected]> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
