Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-06 Thread Renat Akhmerov
Ok, good!

Renat Akhmerov
@ Mirantis Inc.



On 06 Mar 2014, at 15:25, Nikolay Makhotkin  wrote:

> Manas, Renat, no problem :)
> 
> The commit is sent already - https://review.openstack.org/#/c/75888/
> 
> 
> On Thu, Mar 6, 2014 at 12:14 PM, Manas Kelshikar  wrote:
> I agree, let's rename data to spec and unblock the check-in.
> 
> Nikolay - Sorry for the trouble :)
> 
> On Mar 5, 2014 10:17 PM, "Renat Akhmerov"  wrote:
> Alright, good input Manas, appreciate.
> 
> My comments are below...
> 
> On 06 Mar 2014, at 10:47, Manas Kelshikar  wrote:
>> Do we have better ideas on how to work with DSL? A good mental exercise here 
>> would be to imagine that we have more than one DSL, not only YAML but say 
>> XML. How would it change the picture?
>> [Manas] As long as we form an abstraction between the DSL format i.e. 
>> YAML/XML and it consumption we will be able to move between various formats. 
>> (wishful) My personal preference is to not even have DSL show up anywhere in 
>> Mistral code apart from take it as input and transforming into this first 
>> level specification model - I know this is not the current state.
> 
> Totally agree with your point. That’s what we’re trying to achieve.
>> How can we clearly distinguish between these two models so that it wouldn’t 
>> be confusing?
>> Do we have a better naming in mind?
>> [Manas] I think we all would agree that the best approach is to have precise 
>> naming.
>> 
>> I see your point of de-normalizing the dsl data into respective db model 
>> objects.
>> 
>> In a previous email I suggested using *Spec. I will try to build on this -
>> 1. Everything specified via the YAML input is a specification or definition 
>> or template. Therefore I suggest we suffix all these types with 
>> Spec/Definition/Template. So TaskSpec/TaskDefinition/TaskTemplate etc.. As 
>> per the latest change these are TaskData ... ActionData. 
> 
> After all the time I spent thinking about it I would choose Spec suffix since 
> it’s short and expresses the intention well enough. In conjunction with 
> “workbook” package name it would look very nice (basically we get 
> specification of workbook which is what we’re talking about, right?)
> 
> So if you agree then let’s change to TaskSpec, ActionSpec etc. Nikolay, sorry 
> for making you change this patch again and again :) But it’s really important 
> and going to have a long-term effect at the entire system.
> 
>> 2. As per current impl the YAML is stored as a key-value in the DB. This is 
>> fine since that is front-ended by objects that Nikolay has introduced. e.g. 
>> TaskData, ActionData etc.
> 
> Yep, right. The only thing I would suggest is to avoid DB fields like 
> “task_dsl” like we have now. The alternative could be “task_spec”.
> 
>> 3. As per my thinking a model object that ends up in the DB or a model 
>> object that is in memory all can reside in the same module. I view 
>> persistence as an orthogonal concern so no real reason to distinguish the 
>> module names of the two set of models. If we do choose to distinguish as per 
>> latest change i.e. mistral/workbook that works too.
> 
> Sorry, I believe I wasn’t clear enough on this thing. I think we shouldn’t 
> have these two models in the same package since what I meant by “DB model” is 
> actually “execution” and “task” that carry workflow runtime information and 
> refer to a particular execution (we could also call it “session”). So my 
> point is that these are fundamentally different types of models. The best 
> analogy that comes to my mind is the relationship “class -> instance” where 
> in our case “class = Specification" (TaskSpec etc.) and “instance = 
> Execution/Task”. Does it make any sense?
> 
>> @Nikolay - I am generally ok with the approach. I hope that this helps 
>> clarify my thinking and perception. Please ask more questions.
>> 
>> Overall I like the approach of formalizing the 2 models. I am ok with 
>> current state of the review and have laid out my preferences.
> 
> I like the current state of this patch. The only thing I would do is renaming 
> “Data” to “Spec”.
> 
> Thank you.
> 
> Renat Akhmerov
> @ Mirantis Inc.
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> -- 
> Best Regards,
> Nikolay
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-06 Thread Nikolay Makhotkin
Manas, Renat, no problem :)

The commit is sent already - https://review.openstack.org/#/c/75888/


On Thu, Mar 6, 2014 at 12:14 PM, Manas Kelshikar wrote:

> I agree, let's rename data to spec and unblock the check-in.
>
> Nikolay - Sorry for the trouble :)
> On Mar 5, 2014 10:17 PM, "Renat Akhmerov"  wrote:
>
>> Alright, good input Manas, appreciate.
>>
>> My comments are below...
>>
>> On 06 Mar 2014, at 10:47, Manas Kelshikar  wrote:
>>
>>
>>- Do we have better ideas on how to work with DSL? A good mental
>>exercise here would be to imagine that we have more than one DSL, not only
>>YAML but say XML. How would it change the picture?
>>
>> [Manas] As long as we form an abstraction between the DSL format i.e.
>> YAML/XML and it consumption we will be able to move between various
>> formats. (wishful) My personal preference is to not even have DSL show up
>> anywhere in Mistral code apart from take it as input and transforming into
>> this first level specification model - I know this is not the current state.
>>
>>
>> Totally agree with your point. That's what we're trying to achieve.
>>
>>
>>- How can we clearly distinguish between these two models so that it
>>wouldn't be confusing?
>>- Do we have a better naming in mind?
>>
>> [Manas] I think we all would agree that the best approach is to have
>> precise naming.
>>
>> I see your point of de-normalizing the dsl data into respective db model
>> objects.
>>
>> In a previous email I suggested using *Spec. I will try to build on this -
>> 1. Everything specified via the YAML input is a specification or
>> definition or template. Therefore I suggest we suffix all these types with
>> Spec/Definition/Template. So TaskSpec/TaskDefinition/TaskTemplate etc.. As
>> per the latest change these are TaskData ... ActionData.
>>
>>
>> After all the time I spent thinking about it I would choose Spec suffix
>> since it's short and expresses the intention well enough. In conjunction
>> with "workbook" package name it would look very nice (basically we get
>> specification of workbook which is what we're talking about, right?)
>>
>> So if you agree then let's change to TaskSpec, ActionSpec etc. Nikolay,
>> sorry for making you change this patch again and again :) But it's really
>> important and going to have a long-term effect at the entire system.
>>
>> 2. As per current impl the YAML is stored as a key-value in the DB. This
>> is fine since that is front-ended by objects that Nikolay has introduced.
>> e.g. TaskData, ActionData etc.
>>
>>
>> Yep, right. The only thing I would suggest is to avoid DB fields like
>> "task_dsl" like we have now. The alternative could be "task_spec".
>>
>> 3. As per my thinking a model object that ends up in the DB or a model
>> object that is in memory all can reside in the same module. I view
>> persistence as an orthogonal concern so no real reason to distinguish the
>> module names of the two set of models. If we do choose to distinguish as
>> per latest change i.e. mistral/workbook that works too.
>>
>>
>> Sorry, I believe I wasn't clear enough on this thing. I think we
>> shouldn't have these two models in the same package since what I meant by
>> "DB model" is actually "execution" and "task" that carry workflow runtime
>> information and refer to a particular execution (we could also call it
>> "session"). So my point is that these are fundamentally different types of
>> models. The best analogy that comes to my mind is the relationship "class
>> -> instance" where in our case "class = Specification" (TaskSpec etc.) and
>> "instance = Execution/Task". Does it make any sense?
>>
>> @Nikolay - I am generally ok with the approach. I hope that this helps
>> clarify my thinking and perception. Please ask more questions.
>>
>> Overall I like the approach of formalizing the 2 models. I am ok with
>> current state of the review and have laid out my preferences.
>>
>>
>> I like the current state of this patch. The only thing I would do is
>> renaming "Data" to "Spec".
>>
>> Thank you.
>>
>> Renat Akhmerov
>> @ Mirantis Inc.
>>
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-06 Thread Manas Kelshikar
I agree, let's rename data to spec and unblock the check-in.

Nikolay - Sorry for the trouble :)
On Mar 5, 2014 10:17 PM, "Renat Akhmerov"  wrote:

> Alright, good input Manas, appreciate.
>
> My comments are below...
>
> On 06 Mar 2014, at 10:47, Manas Kelshikar  wrote:
>
>
>- Do we have better ideas on how to work with DSL? A good mental
>exercise here would be to imagine that we have more than one DSL, not only
>YAML but say XML. How would it change the picture?
>
> [Manas] As long as we form an abstraction between the DSL format i.e.
> YAML/XML and it consumption we will be able to move between various
> formats. (wishful) My personal preference is to not even have DSL show up
> anywhere in Mistral code apart from take it as input and transforming into
> this first level specification model - I know this is not the current state.
>
>
> Totally agree with your point. That's what we're trying to achieve.
>
>
>- How can we clearly distinguish between these two models so that it
>wouldn't be confusing?
>- Do we have a better naming in mind?
>
> [Manas] I think we all would agree that the best approach is to have
> precise naming.
>
> I see your point of de-normalizing the dsl data into respective db model
> objects.
>
> In a previous email I suggested using *Spec. I will try to build on this -
> 1. Everything specified via the YAML input is a specification or
> definition or template. Therefore I suggest we suffix all these types with
> Spec/Definition/Template. So TaskSpec/TaskDefinition/TaskTemplate etc.. As
> per the latest change these are TaskData ... ActionData.
>
>
> After all the time I spent thinking about it I would choose Spec suffix
> since it's short and expresses the intention well enough. In conjunction
> with "workbook" package name it would look very nice (basically we get
> specification of workbook which is what we're talking about, right?)
>
> So if you agree then let's change to TaskSpec, ActionSpec etc. Nikolay,
> sorry for making you change this patch again and again :) But it's really
> important and going to have a long-term effect at the entire system.
>
> 2. As per current impl the YAML is stored as a key-value in the DB. This
> is fine since that is front-ended by objects that Nikolay has introduced.
> e.g. TaskData, ActionData etc.
>
>
> Yep, right. The only thing I would suggest is to avoid DB fields like
> "task_dsl" like we have now. The alternative could be "task_spec".
>
> 3. As per my thinking a model object that ends up in the DB or a model
> object that is in memory all can reside in the same module. I view
> persistence as an orthogonal concern so no real reason to distinguish the
> module names of the two set of models. If we do choose to distinguish as
> per latest change i.e. mistral/workbook that works too.
>
>
> Sorry, I believe I wasn't clear enough on this thing. I think we shouldn't
> have these two models in the same package since what I meant by "DB model"
> is actually "execution" and "task" that carry workflow runtime information
> and refer to a particular execution (we could also call it "session"). So
> my point is that these are fundamentally different types of models. The
> best analogy that comes to my mind is the relationship "class -> instance"
> where in our case "class = Specification" (TaskSpec etc.) and "instance =
> Execution/Task". Does it make any sense?
>
> @Nikolay - I am generally ok with the approach. I hope that this helps
> clarify my thinking and perception. Please ask more questions.
>
> Overall I like the approach of formalizing the 2 models. I am ok with
> current state of the review and have laid out my preferences.
>
>
> I like the current state of this patch. The only thing I would do is
> renaming "Data" to "Spec".
>
> Thank you.
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-05 Thread Renat Akhmerov
Alright, good input Manas, appreciate.

My comments are below...

On 06 Mar 2014, at 10:47, Manas Kelshikar  wrote:
> Do we have better ideas on how to work with DSL? A good mental exercise here 
> would be to imagine that we have more than one DSL, not only YAML but say 
> XML. How would it change the picture?
> [Manas] As long as we form an abstraction between the DSL format i.e. 
> YAML/XML and it consumption we will be able to move between various formats. 
> (wishful) My personal preference is to not even have DSL show up anywhere in 
> Mistral code apart from take it as input and transforming into this first 
> level specification model - I know this is not the current state.

Totally agree with your point. That’s what we’re trying to achieve.
> How can we clearly distinguish between these two models so that it wouldn’t 
> be confusing?
> Do we have a better naming in mind?
> [Manas] I think we all would agree that the best approach is to have precise 
> naming.
> 
> I see your point of de-normalizing the dsl data into respective db model 
> objects.
> 
> In a previous email I suggested using *Spec. I will try to build on this -
> 1. Everything specified via the YAML input is a specification or definition 
> or template. Therefore I suggest we suffix all these types with 
> Spec/Definition/Template. So TaskSpec/TaskDefinition/TaskTemplate etc.. As 
> per the latest change these are TaskData ... ActionData. 

After all the time I spent thinking about it I would choose Spec suffix since 
it’s short and expresses the intention well enough. In conjunction with 
“workbook” package name it would look very nice (basically we get specification 
of workbook which is what we’re talking about, right?)

So if you agree then let’s change to TaskSpec, ActionSpec etc. Nikolay, sorry 
for making you change this patch again and again :) But it’s really important 
and going to have a long-term effect at the entire system.

> 2. As per current impl the YAML is stored as a key-value in the DB. This is 
> fine since that is front-ended by objects that Nikolay has introduced. e.g. 
> TaskData, ActionData etc.

Yep, right. The only thing I would suggest is to avoid DB fields like 
“task_dsl” like we have now. The alternative could be “task_spec”.

> 3. As per my thinking a model object that ends up in the DB or a model object 
> that is in memory all can reside in the same module. I view persistence as an 
> orthogonal concern so no real reason to distinguish the module names of the 
> two set of models. If we do choose to distinguish as per latest change i.e. 
> mistral/workbook that works too.

Sorry, I believe I wasn’t clear enough on this thing. I think we shouldn’t have 
these two models in the same package since what I meant by “DB model” is 
actually “execution” and “task” that carry workflow runtime information and 
refer to a particular execution (we could also call it “session”). So my point 
is that these are fundamentally different types of models. The best analogy 
that comes to my mind is the relationship “class -> instance” where in our case 
“class = Specification" (TaskSpec etc.) and “instance = Execution/Task”. Does 
it make any sense?

> @Nikolay - I am generally ok with the approach. I hope that this helps 
> clarify my thinking and perception. Please ask more questions.
> 
> Overall I like the approach of formalizing the 2 models. I am ok with current 
> state of the review and have laid out my preferences.

I like the current state of this patch. The only thing I would do is renaming 
“Data” to “Spec”.

Thank you.

Renat Akhmerov
@ Mirantis Inc.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-05 Thread Manas Kelshikar
Renat,

Thanks for the detailed explanation. It is quite instructive and helps
frame the question accurately by providing the necessary context.

   - Is the approach itself reasonable?

[Manas] I think so. A repeatable workflow requires these sort of separation
i.e. a model in which to save the specification as supplied by the user and
a model in which to save the execution state - current and past.

   - Do we have better ideas on how to work with DSL? A good mental
   exercise here would be to imagine that we have more than one DSL, not only
   YAML but say XML. How would it change the picture?

[Manas] As long as we form an abstraction between the DSL format i.e.
YAML/XML and it consumption we will be able to move between various
formats. (wishful) My personal preference is to not even have DSL show up
anywhere in Mistral code apart from take it as input and transforming into
this first level specification model - I know this is not the current state.

   - How can we clearly distinguish between these two models so that it
   wouldn't be confusing?
   - Do we have a better naming in mind?

[Manas] I think we all would agree that the best approach is to have
precise naming.

I see your point of de-normalizing the dsl data into respective db model
objects.

In a previous email I suggested using *Spec. I will try to build on this -
1. Everything specified via the YAML input is a specification or definition
or template. Therefore I suggest we suffix all these types with
Spec/Definition/Template. So TaskSpec/TaskDefinition/TaskTemplate etc.. As
per the latest change these are TaskData ... ActionData.
2. As per current impl the YAML is stored as a key-value in the DB. This is
fine since that is front-ended by objects that Nikolay has introduced. e.g.
TaskData, ActionData etc.
3. As per my thinking a model object that ends up in the DB or a model
object that is in memory all can reside in the same module. I view
persistence as an orthogonal concern so no real reason to distinguish the
module names of the two set of models. If we do choose to distinguish as
per latest change i.e. mistral/workbook that works too.

@Nikolay - I am generally ok with the approach. I hope that this helps
clarify my thinking and perception. Please ask more questions.

Overall I like the approach of formalizing the 2 models. I am ok with
current state of the review and have laid out my preferences.

Thanks,
Manas

On Wed, Mar 5, 2014 at 3:39 AM, Nikolay Makhotkin
wrote:

> I think today and I have a good name for package (instead of
> 'mistral/model')
>
> How do you think about to name it 'mistral/workbook'? I.e., It means that
> it contains modules for work with workbook representation - tasks,
> services, actions and workflow.
>
> This way we able to get rid of any confusing.
>
>
> Best Regards,
> Nikolay
>
>
> On Wed, Mar 5, 2014 at 8:50 AM, Renat Akhmerov wrote:
>
>> I think we forgot to point to the commit itself. Here it is:
>> https://review.openstack.org/#/c/77126/
>>
>> Manas, can you please provide more details on your suggestion?
>>
>> For now let me just describe the background of Nikolay's question.
>>
>> Basically, we are talking about how we are working with data inside
>> Mistral. So far, for example, if a user sent a request to Mistral "start
>> workflow" then Mistral would do the following:
>>
>>- Get workbook DSL (YAML) from the DB (given that it's been already
>>persisted earlier).
>>- Load it into a dictionary-like structure using standard 'yaml'
>>library.
>>- Based on this dictionary-like structure create all necessary DB
>>objects to track the state of workflow execution objects and individual
>>tasks.
>>- Perform all the necessary logic in engine and so on. The important
>>thing here is that DB objects contain corresponding DSL snippets as they
>>are described in DSL (e.g. tasks have property "task_dsl") to reduce the
>>complexity of relational model that we have in DB. Otherwise it would be
>>really complicated and most of the queries would contain lots of joins. 
>> The
>>example of non-trivial relation in DSL is "task"->"action
>>name"->"service"->"service actions"->"action", as you can see it would be
>>hard to navigate to action in the DB from a task if our relational model
>>matches to what we have in DSL. this approach leads to the problem of
>>addressing any dsl properties using hardcoded strings which are spread
>>across the code and that brings lots of pain when doing refactoring, when
>>trying to understand the structure of the model we describe in DSL, it
>>doesn't allow to do validation easily and so on.
>>
>>
>> So what we have in DB we've called "model" so far and we've called just
>> "dsl" the dictionary structure coming from DSL. So if we got a part of the
>> structure related to a task we would call it "dsl_task".
>>
>> So what Nikolay is doing now is he's reworking the approach how we work
>> with DSL. Now we as

Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-05 Thread Nikolay Makhotkin
I think today and I have a good name for package (instead of
'mistral/model')

How do you think about to name it 'mistral/workbook'? I.e., It means that
it contains modules for work with workbook representation - tasks,
services, actions and workflow.

This way we able to get rid of any confusing.


Best Regards,
Nikolay

On Wed, Mar 5, 2014 at 8:50 AM, Renat Akhmerov wrote:

> I think we forgot to point to the commit itself. Here it is:
> https://review.openstack.org/#/c/77126/
>
> Manas, can you please provide more details on your suggestion?
>
> For now let me just describe the background of Nikolay's question.
>
> Basically, we are talking about how we are working with data inside
> Mistral. So far, for example, if a user sent a request to Mistral "start
> workflow" then Mistral would do the following:
>
>- Get workbook DSL (YAML) from the DB (given that it's been already
>persisted earlier).
>- Load it into a dictionary-like structure using standard 'yaml'
>library.
>- Based on this dictionary-like structure create all necessary DB
>objects to track the state of workflow execution objects and individual
>tasks.
>- Perform all the necessary logic in engine and so on. The important
>thing here is that DB objects contain corresponding DSL snippets as they
>are described in DSL (e.g. tasks have property "task_dsl") to reduce the
>complexity of relational model that we have in DB. Otherwise it would be
>really complicated and most of the queries would contain lots of joins. The
>example of non-trivial relation in DSL is "task"->"action
>name"->"service"->"service actions"->"action", as you can see it would be
>hard to navigate to action in the DB from a task if our relational model
>matches to what we have in DSL. this approach leads to the problem of
>addressing any dsl properties using hardcoded strings which are spread
>across the code and that brings lots of pain when doing refactoring, when
>trying to understand the structure of the model we describe in DSL, it
>doesn't allow to do validation easily and so on.
>
>
> So what we have in DB we've called "model" so far and we've called just
> "dsl" the dictionary structure coming from DSL. So if we got a part of the
> structure related to a task we would call it "dsl_task".
>
> So what Nikolay is doing now is he's reworking the approach how we work
> with DSL. Now we assume that after we parsed a workbook DSL we get some
> "model". So that we don't use "dsl" in the code anywhere this "model"
> describes basically the structure of what we have in DSL and that would
> allow to address the problems I mentioned above (hardcoded strings are
> replaced with access methods, we clearly see the structure of what we're
> working with, we can easily validate it and so on). So when we need to
> access some DSL properties we would need to get workbook DSL from DB, build
> this model out of it and continue to work with it.
>
> Long story short, this model parsed from DSL is not the model we store in
> DB but they're both called "model" which may be confusing. For me this
> non-DB model more looks like "domain model" or something like this. So the
> question I would ask ourselves here:
>
>- Is the approach itself reasonable?
>- Do we have better ideas on how to work with DSL? A good mental
>exercise here would be to imagine that we have more than one DSL, not only
>YAML but say XML. How would it change the picture?
>- How can we clearly distinguish between these two models so that it
>wouldn't be confusing?
>- Do we have a better naming in mind?
>
>
> Thanks.
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
> On 05 Mar 2014, at 08:56, Manas Kelshikar  wrote:
>
> Since the renaming is for types in mistral.model.*. I am thinking we
> suffix with Spec e.g.
>
> TaskObject -> TaskSpec
> ActionObject -> ActionSpec and so on.
>
> The "Spec" suggest that it is a specification of the final object that
> ends up in the DB and not the actual object. Multiple actual objects can be
> derived from these Spec objects which fits well with the current paradigm.
> Thoughts?
>
>
> On Mon, Mar 3, 2014 at 9:43 PM, Manas Kelshikar wrote:
>
>> Hi Nikolay -
>>
>> Is your concern that mistral.db.sqlalchemy.models.* and mistral.model.*
>> will lead to confusion or something else?
>>
>> IMHO as per your change model seems like the appropriate usage while what
>> is stored in the DB is also a model. If we pick appropriate names to
>> distinguish between nature of the objects we should be able to avoid any
>> confusion and whether or not model appears in the module name should not
>> matter much.
>>
>> Thanks,
>> Manas
>>
>>
>> On Mon, Mar 3, 2014 at 8:43 AM, Nikolay Makhotkin <
>> nmakhot...@mirantis.com> wrote:
>>
>>> Hi, team!
>>>
>>> Please look at the commit .
>>>
>>> Module 'mistral/model' now is responsible for object model
>>> representation which is used for accessing properties of actions

Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-04 Thread Renat Akhmerov
I think we forgot to point to the commit itself. Here it is: 
https://review.openstack.org/#/c/77126/

Manas, can you please provide more details on your suggestion?

For now let me just describe the background of Nikolay’s question.

Basically, we are talking about how we are working with data inside Mistral. So 
far, for example, if a user sent a request to Mistral “start workflow” then 
Mistral would do the following:
Get workbook DSL (YAML) from the DB (given that it’s been already persisted 
earlier).
Load it into a dictionary-like structure using standard ‘yaml’ library.
Based on this dictionary-like structure create all necessary DB objects to 
track the state of workflow execution objects and individual tasks.
Perform all the necessary logic in engine and so on. The important thing here 
is that DB objects contain corresponding DSL snippets as they are described in 
DSL (e.g. tasks have property “task_dsl”) to reduce the complexity of 
relational model that we have in DB. Otherwise it would be really complicated 
and most of the queries would contain lots of joins. The example of non-trivial 
relation in DSL is “task”->”action name”->”service”->”service 
actions”->”action”, as you can see it would be hard to navigate to action in 
the DB from a task if our relational model matches to what we have in DSL. this 
approach leads to the problem of addressing any dsl properties using hardcoded 
strings which are spread across the code and that brings lots of pain when 
doing refactoring, when trying to understand the structure of the model we 
describe in DSL, it doesn’t allow to do validation easily and so on.

So what we have in DB we’ve called “model” so far and we’ve called just “dsl” 
the dictionary structure coming from DSL. So if we got a part of the structure 
related to a task we would call it “dsl_task”.

So what Nikolay is doing now is he’s reworking the approach how we work with 
DSL. Now we assume that after we parsed a workbook DSL we get some “model”. So 
that we don’t use “dsl” in the code anywhere this “model” describes basically 
the structure of what we have in DSL and that would allow to address the 
problems I mentioned above (hardcoded strings are replaced with access methods, 
we clearly see the structure of what we’re working with, we can easily validate 
it and so on). So when we need to access some DSL properties we would need to 
get workbook DSL from DB, build this model out of it and continue to work with 
it.

Long story short, this model parsed from DSL is not the model we store in DB 
but they’re both called “model” which may be confusing. For me this non-DB 
model more looks like “domain model” or something like this. So the question I 
would ask ourselves here:
Is the approach itself reasonable?
Do we have better ideas on how to work with DSL? A good mental exercise here 
would be to imagine that we have more than one DSL, not only YAML but say XML. 
How would it change the picture?
How can we clearly distinguish between these two models so that it wouldn’t be 
confusing?
Do we have a better naming in mind?

Thanks.

Renat Akhmerov
@ Mirantis Inc.



On 05 Mar 2014, at 08:56, Manas Kelshikar  wrote:

> Since the renaming is for types in mistral.model.*. I am thinking we suffix 
> with Spec e.g. 
> 
> TaskObject -> TaskSpec
> ActionObject -> ActionSpec and so on.
> 
> The "Spec" suggest that it is a specification of the final object that ends 
> up in the DB and not the actual object. Multiple actual objects can be 
> derived from these Spec objects which fits well with the current paradigm. 
> Thoughts?
> 
> 
> On Mon, Mar 3, 2014 at 9:43 PM, Manas Kelshikar  wrote:
> Hi Nikolay - 
> 
> Is your concern that mistral.db.sqlalchemy.models.* and mistral.model.* will 
> lead to confusion or something else? 
> 
> IMHO as per your change model seems like the appropriate usage while what is 
> stored in the DB is also a model. If we pick appropriate names to distinguish 
> between nature of the objects we should be able to avoid any confusion and 
> whether or not model appears in the module name should not matter much.
> 
> Thanks,
> Manas
> 
> 
> On Mon, Mar 3, 2014 at 8:43 AM, Nikolay Makhotkin  
> wrote:
> Hi, team! 
> 
> Please look at the commit .
> 
> Module 'mistral/model' now is responsible for object model representation 
> which is used for accessing properties of actions, tasks etc.
> 
> We have a name problem - looks like we should rename module 'mistral/model' 
> since we have DB models and they are absolutely different.
> 
> 
> Thoughts?
> 
>  
> Best Regards,
> Nikolay
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__

Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-04 Thread Manas Kelshikar
Since the renaming is for types in mistral.model.*. I am thinking we suffix
with Spec e.g.

TaskObject -> TaskSpec
ActionObject -> ActionSpec and so on.

The "Spec" suggest that it is a specification of the final object that ends
up in the DB and not the actual object. Multiple actual objects can be
derived from these Spec objects which fits well with the current paradigm.
Thoughts?


On Mon, Mar 3, 2014 at 9:43 PM, Manas Kelshikar wrote:

> Hi Nikolay -
>
> Is your concern that mistral.db.sqlalchemy.models.* and mistral.model.*
> will lead to confusion or something else?
>
> IMHO as per your change model seems like the appropriate usage while what
> is stored in the DB is also a model. If we pick appropriate names to
> distinguish between nature of the objects we should be able to avoid any
> confusion and whether or not model appears in the module name should not
> matter much.
>
> Thanks,
> Manas
>
>
> On Mon, Mar 3, 2014 at 8:43 AM, Nikolay Makhotkin  > wrote:
>
>> Hi, team!
>>
>> Please look at the commit .
>>
>> Module 'mistral/model' now is responsible for object model representation
>> which is used for accessing properties of actions, tasks etc.
>>
>> We have a name problem - looks like we should rename module
>> 'mistral/model' since we have DB models and they are absolutely different.
>>
>>
>> Thoughts?
>>
>>
>> Best Regards,
>> Nikolay
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-03 Thread Manas Kelshikar
Hi Nikolay -

Is your concern that mistral.db.sqlalchemy.models.* and mistral.model.*
will lead to confusion or something else?

IMHO as per your change model seems like the appropriate usage while what
is stored in the DB is also a model. If we pick appropriate names to
distinguish between nature of the objects we should be able to avoid any
confusion and whether or not model appears in the module name should not
matter much.

Thanks,
Manas


On Mon, Mar 3, 2014 at 8:43 AM, Nikolay Makhotkin
wrote:

> Hi, team!
>
> Please look at the commit .
>
> Module 'mistral/model' now is responsible for object model representation
> which is used for accessing properties of actions, tasks etc.
>
> We have a name problem - looks like we should rename module
> 'mistral/model' since we have DB models and they are absolutely different.
>
>
> Thoughts?
>
>
> Best Regards,
> Nikolay
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-03 Thread Nikolay Makhotkin
Hi, team!

Please look at the commit .

Module 'mistral/model' now is responsible for object model representation
which is used for accessing properties of actions, tasks etc.

We have a name problem - looks like we should rename module 'mistral/model'
since we have DB models and they are absolutely different.


Thoughts?


Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev