[jira] [Updated] (ARIA-250) Sporadic Appveyor failures even if all tests pass

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIA-250:
-
Description: 
Sometimes, even if all the tests pass, we still get a failure from Appveyor:
{code}
ERROR: InvocationError: 
'C:\\projects\\incubator-ariatosca\\.tox\\pywin\\Scripts\\pytest.EXE tests 
--ignore=tests/end2end --cov-report term-missing --cov aria'
{code}

For reference, see:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1484
compared to:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1485

  was:
Sometimes, even if all the tests pass, we still get a failure from Appveyor:
{{ERROR: InvocationError: 
'C:\\projects\\incubator-ariatosca\\.tox\\pywin\\Scripts\\pytest.EXE tests 
--ignore=tests/end2end --cov-report term-missing --cov aria'}}

For reference, see:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1484
compared to:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1485


> Sporadic Appveyor failures even if all tests pass
> -
>
> Key: ARIA-250
> URL: https://issues.apache.org/jira/browse/ARIA-250
> Project: AriaTosca
>  Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Avia Efrat
>Priority: Minor
>
> Sometimes, even if all the tests pass, we still get a failure from Appveyor:
> {code}
> ERROR: InvocationError: 
> 'C:\\projects\\incubator-ariatosca\\.tox\\pywin\\Scripts\\pytest.EXE tests 
> --ignore=tests/end2end --cov-report term-missing --cov aria'
> {code}
> For reference, see:
> https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1484
> compared to:
> https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1485



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (ARIA-171) Service_template.services type inconsistency

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv reassigned ARIA-171:


Assignee: Avia Efrat  (was: Ran Ziv)

> Service_template.services type inconsistency
> 
>
> Key: ARIA-171
> URL: https://issues.apache.org/jira/browse/ARIA-171
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> {{service_template.services}} is an object of type {{InstrumentedList}}, 
> which is inconsistent with similar forms of model access (e.g. 
> {{service.nodes}}).
> The object's type should instead be a {{MappedCollection}} from the service 
> name to the service model object.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ARIA-253) HOST keyword in intrinsic functions uses only the compute host node

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIA-253:
-
Issue Type: Story  (was: Task)

> HOST keyword in intrinsic functions uses only the compute host node
> ---
>
> Key: ARIA-253
> URL: https://issues.apache.org/jira/browse/ARIA-253
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Priority: Minor
>
> The {{HOST}} keyword (which may be used in the {{get_property}} and the 
> {{get_attribute}} intrinsic functions) will look for the desired value at the 
> {{compute}} host node of the current node.
> However, according to the TOSCA spec, it seems that the value should instead 
> be looked over at all containing host nodes - and not necessarily only the 
> topmost {{compute}} host.
> This should be looked further into, as this can also cause ambiguity in some 
> cases.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ARIA-253) HOST keyword in intrinsic functions uses only the compute host node

[Ran Ziv (JIRA)]

Ran Ziv created ARIA-253:


 Summary: HOST keyword in intrinsic functions uses only the compute 
host node
 Key: ARIA-253
 URL: https://issues.apache.org/jira/browse/ARIA-253
 Project: AriaTosca
  Issue Type: Task
Reporter: Ran Ziv
Priority: Minor


The {{HOST}} keyword (which may be used in the {{get_property}} and the 
{{get_attribute}} intrinsic functions) will look for the desired value at the 
{{compute}} host node of the current node.

However, according to the TOSCA spec, it seems that the value should instead be 
looked over at all containing host nodes - and not necessarily only the topmost 
{{compute}} host.

This should be looked further into, as this can also cause ambiguity in some 
cases.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ARIA-251) service/service_template columns for Parameter model

[Ran Ziv (JIRA)]

Ran Ziv created ARIA-251:


 Summary: service/service_template columns for Parameter model
 Key: ARIA-251
 URL: https://issues.apache.org/jira/browse/ARIA-251
 Project: AriaTosca
  Issue Type: Task
Reporter: Ran Ziv
Priority: Minor


The {{Parameter}} model has the {{service}} and {{service_template}} fields 
which are both calculated rather than stored.
It's worth considering having these two as persistent values (i.e. columns) for 
this model.

The main downside is that it'd require having and passing this information 
wherever parameters are created - however these two should be relatively 
readily accessible (unlike {{owner}} / {{container}} (?))




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ARIA-118) plugin.yaml importing

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIA-118:
-
Description: 
Using a plugin currently requires a user first installs the plugin (using 
PluginManager), then import the relevant plugin.yaml file in the service 
template file. The import will currently likely point to a URL, or be a path 
relative to the service-template yaml file.

Some ideas for improvement and easing the import;
 - If a plugin contained its plugin.yaml as part of its wagon archive, then 
once installed, users could import the yaml file more easily using a notation 
such as {{plugins/openstack.yaml}} (or perhaps {{openstack.yaml}}, having the 
import mechanism iterate over plugins looking for this resource file or so)
 - The import mechanism could look for imports in the resource-storage as well 
- There could be a directory on the resource-storage designated for storing 
global yaml files for import, thereby simplifying reuse of yaml imports across 
service-templates.
 - Perhaps ARIA should also support importing yaml files by using paths 
relative to the service-template's package root (as opposed to only looking for 
paths relative to the current yaml file)? Note that this could lead to 
ambiguities in some cases.


Note that the last two don't necessarily have to do with plugins directly, but 
it's more likely to be relevant for plugins as they're used across 
service-templates more often.


  was:
Using a plugin currently requires a user first installs the plugin (using 
PluginManager), then import the relevant plugin.yaml file in the service 
template file. The import will currently likely to point to a URL.

It should be simpler if a plugin contained its plugin.yaml as part of its wagon 
archive, and once installed, users should be able to import it more easily 
using a notation such as `plugins/openstack.yaml` (or perhaps `openstack.yaml` 
and have a fallback when importing that would iterate all plugins etc...)


> plugin.yaml importing
> -
>
> Key: ARIA-118
> URL: https://issues.apache.org/jira/browse/ARIA-118
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Priority: Minor
>
> Using a plugin currently requires a user first installs the plugin (using 
> PluginManager), then import the relevant plugin.yaml file in the service 
> template file. The import will currently likely point to a URL, or be a path 
> relative to the service-template yaml file.
> Some ideas for improvement and easing the import;
>  - If a plugin contained its plugin.yaml as part of its wagon archive, then 
> once installed, users could import the yaml file more easily using a notation 
> such as {{plugins/openstack.yaml}} (or perhaps {{openstack.yaml}}, having the 
> import mechanism iterate over plugins looking for this resource file or so)
>  - The import mechanism could look for imports in the resource-storage as 
> well - There could be a directory on the resource-storage designated for 
> storing global yaml files for import, thereby simplifying reuse of yaml 
> imports across service-templates.
>  - Perhaps ARIA should also support importing yaml files by using paths 
> relative to the service-template's package root (as opposed to only looking 
> for paths relative to the current yaml file)? Note that this could lead to 
> ambiguities in some cases.
> Note that the last two don't necessarily have to do with plugins directly, 
> but it's more likely to be relevant for plugins as they're used across 
> service-templates more often.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ARIA-118) plugin.yaml importing

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIA-118:
-
Description: 
Using a plugin currently requires a user first installs the plugin (using 
PluginManager), then import the relevant plugin.yaml file in the service 
template file. The import will currently likely point to a URL, or be a path 
relative to the service-template yaml file.

Some ideas for improvement and easing the import;
 - If a plugin contained its plugin.yaml as part of its wagon archive, then 
once installed, users could import the yaml file more easily using a notation 
such as {{plugins/openstack.yaml}} (or perhaps {{openstack.yaml}}, having the 
import mechanism iterate over plugins looking for this resource file or so)

 - The import mechanism could look for imports in the resource-storage as well 
- There could be a directory on the resource-storage designated for storing 
global yaml files for import, thereby simplifying reuse of yaml imports across 
service-templates.

 - Perhaps ARIA should also support importing yaml files by using paths 
relative to the service-template's package root (as opposed to only looking for 
paths relative to the current yaml file)? Note that this could lead to 
ambiguities in some cases.


Note that the last two don't necessarily have to do with plugins directly, but 
it's more likely to be relevant for plugins as they're used across 
service-templates more often.


  was:
Using a plugin currently requires a user first installs the plugin (using 
PluginManager), then import the relevant plugin.yaml file in the service 
template file. The import will currently likely point to a URL, or be a path 
relative to the service-template yaml file.

Some ideas for improvement and easing the import;
 - If a plugin contained its plugin.yaml as part of its wagon archive, then 
once installed, users could import the yaml file more easily using a notation 
such as {{plugins/openstack.yaml}} (or perhaps {{openstack.yaml}}, having the 
import mechanism iterate over plugins looking for this resource file or so)
 - The import mechanism could look for imports in the resource-storage as well 
- There could be a directory on the resource-storage designated for storing 
global yaml files for import, thereby simplifying reuse of yaml imports across 
service-templates.
 - Perhaps ARIA should also support importing yaml files by using paths 
relative to the service-template's package root (as opposed to only looking for 
paths relative to the current yaml file)? Note that this could lead to 
ambiguities in some cases.


Note that the last two don't necessarily have to do with plugins directly, but 
it's more likely to be relevant for plugins as they're used across 
service-templates more often.



> plugin.yaml importing
> -
>
> Key: ARIA-118
> URL: https://issues.apache.org/jira/browse/ARIA-118
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Priority: Minor
>
> Using a plugin currently requires a user first installs the plugin (using 
> PluginManager), then import the relevant plugin.yaml file in the service 
> template file. The import will currently likely point to a URL, or be a path 
> relative to the service-template yaml file.
> Some ideas for improvement and easing the import;
>  - If a plugin contained its plugin.yaml as part of its wagon archive, then 
> once installed, users could import the yaml file more easily using a notation 
> such as {{plugins/openstack.yaml}} (or perhaps {{openstack.yaml}}, having the 
> import mechanism iterate over plugins looking for this resource file or so)
>  - The import mechanism could look for imports in the resource-storage as 
> well - There could be a directory on the resource-storage designated for 
> storing global yaml files for import, thereby simplifying reuse of yaml 
> imports across service-templates.
>  - Perhaps ARIA should also support importing yaml files by using paths 
> relative to the service-template's package root (as opposed to only looking 
> for paths relative to the current yaml file)? Note that this could lead to 
> ambiguities in some cases.
> Note that the last two don't necessarily have to do with plugins directly, 
> but it's more likely to be relevant for plugins as they're used across 
> service-templates more often.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[incubator-ariatosca] Git Push Summary

[avia]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-157-cli-service-template-store-tests-fail-in-windows 
[deleted] 6864d42e1

[jira] [Created] (ARIA-250) Sporadic Appveyor failures even if all tests pass

[Avia Efrat (JIRA)]

Avia Efrat created ARIA-250:
---

 Summary: Sporadic Appveyor failures even if all tests pass
 Key: ARIA-250
 URL: https://issues.apache.org/jira/browse/ARIA-250
 Project: AriaTosca
  Issue Type: Bug
Affects Versions: 0.1.0
Reporter: Avia Efrat
Priority: Minor


Sometimes, even if all the tests pass, we still get a failure from Appveyor:
{{ERROR: InvocationError: 
'C:\\projects\\incubator-ariatosca\\.tox\\pywin\\Scripts\\pytest.EXE tests 
--ignore=tests/end2end --cov-report term-missing --cov aria'}}

For reference, see:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1484
compared to:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-ariatosca/build/1.0.1485



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (ARIA-148) Enhance CLI "show" commands

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv resolved ARIA-148.
--
   Resolution: Fixed
Fix Version/s: 0.1.0

> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
> Fix For: 0.1.0
>
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (ARIA-148) Enhance CLI "show" commands

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv closed ARIA-148.


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
> Fix For: 0.1.0
>
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (ARIA-186) "aria services show" CLI command

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv reopened ARIA-186:
--

> "aria services show" CLI command
> 
>
> Key: ARIA-186
> URL: https://issues.apache.org/jira/browse/ARIA-186
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>Priority: Minor
> Fix For: 0.1.0
>
>
> {{services}} is the only CLI command which doesn't have a {{show}} subcommand.
> There should be one with similar output format to that of the other {{show}} 
> subcommands.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (ARIA-186) "aria services show" CLI command

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv resolved ARIA-186.
--
   Resolution: Fixed
Fix Version/s: 0.1.0

> "aria services show" CLI command
> 
>
> Key: ARIA-186
> URL: https://issues.apache.org/jira/browse/ARIA-186
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>Priority: Minor
> Fix For: 0.1.0
>
>
> {{services}} is the only CLI command which doesn't have a {{show}} subcommand.
> There should be one with similar output format to that of the other {{show}} 
> subcommands.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (ARIA-148) Enhance CLI "show" commands

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv reopened ARIA-148:
--

> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (ARIA-186) "aria services show" CLI command

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv closed ARIA-186.


> "aria services show" CLI command
> 
>
> Key: ARIA-186
> URL: https://issues.apache.org/jira/browse/ARIA-186
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>Priority: Minor
> Fix For: 0.1.0
>
>
> {{services}} is the only CLI command which doesn't have a {{show}} subcommand.
> There should be one with similar output format to that of the other {{show}} 
> subcommands.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[incubator-ariatosca] Git Push Summary

[emblemparade]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-140-version-utils [deleted] eae44d0b0

[incubator-ariatosca] Git Push Summary

[emblemparade]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-148-extra-cli-commands [deleted] fdd57c47a

[jira] [Closed] (ARIA-186) "aria services show" CLI command

[Tal Liron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tal Liron closed ARIA-186.
--
Resolution: Done

> "aria services show" CLI command
> 
>
> Key: ARIA-186
> URL: https://issues.apache.org/jira/browse/ARIA-186
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>Priority: Minor
>
> {{services}} is the only CLI command which doesn't have a {{show}} subcommand.
> There should be one with similar output format to that of the other {{show}} 
> subcommands.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-186) "aria services show" CLI command

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007044#comment-16007044
 ] 

ASF subversion and git services commented on ARIA-186:
--

Commit fdd57c47acfe63dc25c6f20bcb3785225b3f774d in incubator-ariatosca's branch 
refs/heads/master from [~emblemparade]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=fdd57c4 ]

ARIA-148 Enhance CLI show commands

* Allow "--full" flag to provide a complete dump
* Allow "--json" and "--yaml" flags for dump in those formats
* Support for node graph and type hierarchies
* Some fixes for YAML dump for our custom types
* Also closes ARIA-186: "aria services show" command


> "aria services show" CLI command
> 
>
> Key: ARIA-186
> URL: https://issues.apache.org/jira/browse/ARIA-186
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>Priority: Minor
>
> {{services}} is the only CLI command which doesn't have a {{show}} subcommand.
> There should be one with similar output format to that of the other {{show}} 
> subcommands.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007043#comment-16007043
 ] 

ASF subversion and git services commented on ARIA-148:
--

Commit fdd57c47acfe63dc25c6f20bcb3785225b3f774d in incubator-ariatosca's branch 
refs/heads/master from [~emblemparade]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=fdd57c4 ]

ARIA-148 Enhance CLI show commands

* Allow "--full" flag to provide a complete dump
* Allow "--json" and "--yaml" flags for dump in those formats
* Support for node graph and type hierarchies
* Some fixes for YAML dump for our custom types
* Also closes ARIA-186: "aria services show" command


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (ARIA-148) Enhance CLI "show" commands

[Tal Liron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tal Liron closed ARIA-148.
--
Resolution: Done

> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-ariatosca pull request #107: ARIA-148 CLI display commands

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/107


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

incubator-ariatosca git commit: ARIA-148 Enhance CLI show commands

[emblemparade]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/master 60ea3ebb2 -> fdd57c47a


ARIA-148 Enhance CLI show commands

* Allow "--full" flag to provide a complete dump
* Allow "--json" and "--yaml" flags for dump in those formats
* Support for node graph and type hierarchies
* Some fixes for YAML dump for our custom types
* Also closes ARIA-186: "aria services show" command


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/fdd57c47
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/fdd57c47
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/fdd57c47

Branch: refs/heads/master
Commit: fdd57c47acfe63dc25c6f20bcb3785225b3f774d
Parents: 60ea3eb
Author: Tal Liron 
Authored: Thu Apr 20 17:54:47 2017 -0500
Committer: Tal Liron 
Committed: Thu May 11 12:42:40 2017 -0500

--
 aria/cli/commands/service_templates.py |  72 ++
 aria/cli/commands/services.py  |  51 ++-
 aria/cli/core/aria.py  | 203 +++-
 aria/cli/helptexts.py  |  12 +-
 aria/cli/table.py  |   3 +-
 aria/modeling/service_instance.py  |   4 +-
 aria/modeling/service_template.py  |   2 +-
 aria/modeling/types.py |   5 +
 aria/utils/collections.py  |   2 +-
 9 files changed, 260 insertions(+), 94 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/fdd57c47/aria/cli/commands/service_templates.py
--
diff --git a/aria/cli/commands/service_templates.py 
b/aria/cli/commands/service_templates.py
index e459871..0a24907 100644
--- a/aria/cli/commands/service_templates.py
+++ b/aria/cli/commands/service_templates.py
@@ -23,11 +23,13 @@ from .. import utils
 from ..core import aria
 from ...core import Core
 from ...storage import exceptions as storage_exceptions
+from ...parser import consumption
+from ...utils import (formatting, collections, console)
 
 
 DESCRIPTION_FIELD_LENGTH_LIMIT = 20
 SERVICE_TEMPLATE_COLUMNS = \
-['id', 'name', 'description', 'main_file_name', 'created_at', 'updated_at']
+('id', 'name', 'description', 'main_file_name', 'created_at', 'updated_at')
 
 
 @aria.group(name='service-templates')
@@ -43,32 +45,52 @@ def service_templates():
 @aria.argument('service-template-name')
 @aria.options.verbose()
 @aria.pass_model_storage
+@aria.options.service_template_mode_full
+@aria.options.mode_types
+@aria.options.format_json
+@aria.options.format_yaml
 @aria.pass_logger
-def show(service_template_name, model_storage, logger):
-"""Show information for a specific service templates
+def show(service_template_name, model_storage, mode_full, mode_types, 
format_json, format_yaml,
+ logger):
+"""Show information for a specific service template
 
 `SERVICE_TEMPLATE_NAME` is the name of the service template to show 
information on.
 """
-logger.info('Showing service template 
{0}...'.format(service_template_name))
 service_template = 
model_storage.service_template.get_by_name(service_template_name)
-service_template_dict = service_template.to_dict()
-service_template_dict['#services'] = len(service_template.services)
 
-column_formatters = \
-
dict(description=table.trim_formatter_generator(DESCRIPTION_FIELD_LENGTH_LIMIT))
-columns = SERVICE_TEMPLATE_COLUMNS + ['#services']
-table.print_data(columns, service_template_dict, 'Service-template:',
- column_formatters=column_formatters, col_max_width=50)
-
-if service_template_dict['description'] is not None:
-logger.info('Description:')
-
logger.info('{0}{1}'.format(service_template_dict['description'].encode('UTF-8')
 or '',
-os.linesep))
-
-if service_template.services:
-logger.info('Existing services:')
-for service in service_template.services:
-logger.info('\t{0}'.format(service.name))
+if format_json or format_yaml:
+mode_full = True
+
+if mode_full:
+consumption.ConsumptionContext()
+if format_json:
+
console.puts(formatting.json_dumps(collections.prune(service_template.as_raw)))
+elif format_yaml:
+
console.puts(formatting.yaml_dumps(collections.prune(service_template.as_raw)))
+else:
+service_template.dump()
+elif mode_types:
+consumption.ConsumptionContext()
+service_template.dump_types()
+else:
+logger.info('Showing service template 
{0}...'.format(service_template_name))
+service_template_dict = service_template.to_dict()
+

[jira] [Assigned] (ARIA-186) "aria services show" CLI command

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv reassigned ARIA-186:


Assignee: Tal Liron

> "aria services show" CLI command
> 
>
> Key: ARIA-186
> URL: https://issues.apache.org/jira/browse/ARIA-186
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>Priority: Minor
>
> {{services}} is the only CLI command which doesn't have a {{show}} subcommand.
> There should be one with similar output format to that of the other {{show}} 
> subcommands.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (ARIA-85) get_attribute intrinsic function

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv resolved ARIA-85.
-
Resolution: Duplicate

> get_attribute intrinsic function
> 
>
> Key: ARIA-85
> URL: https://issues.apache.org/jira/browse/ARIA-85
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>
> Implement a "get_attribute" intrinsic function, for retrieving attributes / 
> runtime-properties from within the blueprint



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (ARIA-85) get_attribute intrinsic function

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv closed ARIA-85.
---

> get_attribute intrinsic function
> 
>
> Key: ARIA-85
> URL: https://issues.apache.org/jira/browse/ARIA-85
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>
> Implement a "get_attribute" intrinsic function, for retrieving attributes / 
> runtime-properties from within the blueprint



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (ARIA-85) get_attribute intrinsic function

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv reopened ARIA-85:
-

> get_attribute intrinsic function
> 
>
> Key: ARIA-85
> URL: https://issues.apache.org/jira/browse/ARIA-85
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>
> Implement a "get_attribute" intrinsic function, for retrieving attributes / 
> runtime-properties from within the blueprint



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (ARIA-85) get_attribute intrinsic function

[Tal Liron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tal Liron closed ARIA-85.
-
Resolution: Done

> get_attribute intrinsic function
> 
>
> Key: ARIA-85
> URL: https://issues.apache.org/jira/browse/ARIA-85
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Tal Liron
>
> Implement a "get_attribute" intrinsic function, for retrieving attributes / 
> runtime-properties from within the blueprint



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (ARIA-139) TOSCA attributes

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv reopened ARIA-139:
--

> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
> Fix For: 0.1.0
>
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (ARIA-139) TOSCA attributes

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv resolved ARIA-139.
--
   Resolution: Fixed
Fix Version/s: 0.1.0

> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
> Fix For: 0.1.0
>
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[1/6] incubator-ariatosca git commit: ARIA-157 Failing CLI service-templates store tests on Windows [Forced Update!]

[emblemparade]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-148-extra-cli-commands 82193541d -> fdd57c47a (forced update)


ARIA-157 Failing CLI service-templates store tests on Windows

Three tests from `aria service-templates store` failed on Windows, but
not on Linux.

The reason for this failures was differing implementation of
os.path.dirname across the platforms.

Python implements os.path.dirname in the ntpath module. There, somewhere
down the line of calls, (a part of) the argument of dirname is tested
for membership in a string (using `in`). In these three tests, the
argument of dirname is of type MagicMock, and an error is raised since
only a string can be tested for membership in a string.

The solution was to mock the dirname calls.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/6864d42e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/6864d42e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/6864d42e

Branch: refs/heads/ARIA-148-extra-cli-commands
Commit: 6864d42e1748b5cd8ecc596fc1b4adc04b0cc9ba
Parents: 0af9e63
Author: Avia Efrat 
Authored: Wed May 10 17:39:49 2017 +0300
Committer: Avia Efrat 
Committed: Wed May 10 18:13:02 2017 +0300

--
 tests/cli/test_service_templates.py | 3 +++
 1 file changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/6864d42e/tests/cli/test_service_templates.py
--
diff --git a/tests/cli/test_service_templates.py 
b/tests/cli/test_service_templates.py
index 22a8fc8..bc3c751 100644
--- a/tests/cli/test_service_templates.py
+++ b/tests/cli/test_service_templates.py
@@ -128,6 +128,7 @@ class TestServiceTemplatesStore(TestCliBase):
 
 monkeypatch.setattr(Core, 'create_service_template', mock_object)
 monkeypatch.setattr(service_template_utils, 'get', mock_object)
+monkeypatch.setattr(os.path, 'dirname', mock_object)
 self.invoke('service_templates store stubpath {name}'.format(
 name=mock_models.SERVICE_TEMPLATE_NAME))
 assert 'Service template {name} stored'.format(
@@ -152,6 +153,7 @@ class TestServiceTemplatesStore(TestCliBase):
 'create_service_template',
 raise_exception(storage_exceptions.NotFoundError,
 msg='UNIQUE constraint failed'))
+monkeypatch.setattr(os.path, 'dirname', mock_object)
 
 assert_exception_raised(
 self.invoke('service_templates store stubpath test_st'),
@@ -164,6 +166,7 @@ class TestServiceTemplatesStore(TestCliBase):
 monkeypatch.setattr(Core,
 'create_service_template',
 raise_exception(storage_exceptions.NotFoundError))
+monkeypatch.setattr(os.path, 'dirname', mock_object)
 
 assert_exception_raised(
 self.invoke('service_templates store stubpath test_st'),

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006891#comment-16006891
 ] 

ASF subversion and git services commented on ARIA-213:
--

Commit 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589 in incubator-ariatosca's branch 
refs/heads/ARIA-148-extra-cli-commands from [~Mxmrlv]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=2ee06b8 ]

ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def 

[jira] [Commented] (ARIA-186) "aria services show" CLI command

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006894#comment-16006894
 ] 

ASF subversion and git services commented on ARIA-186:
--

Commit fdd57c47acfe63dc25c6f20bcb3785225b3f774d in incubator-ariatosca's branch 
refs/heads/ARIA-148-extra-cli-commands from [~emblemparade]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=fdd57c4 ]

ARIA-148 Enhance CLI show commands

* Allow "--full" flag to provide a complete dump
* Allow "--json" and "--yaml" flags for dump in those formats
* Support for node graph and type hierarchies
* Some fixes for YAML dump for our custom types
* Also closes ARIA-186: "aria services show" command


> "aria services show" CLI command
> 
>
> Key: ARIA-186
> URL: https://issues.apache.org/jira/browse/ARIA-186
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Priority: Minor
>
> {{services}} is the only CLI command which doesn't have a {{show}} subcommand.
> There should be one with similar output format to that of the other {{show}} 
> subcommands.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-139) TOSCA attributes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006892#comment-16006892
 ] 

ASF subversion and git services commented on ARIA-139:
--

Commit 60ea3ebb21e762d36115db26563a93dd3cb72003 in incubator-ariatosca's branch 
refs/heads/ARIA-148-extra-cli-commands from [~emblemparade]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=60ea3eb ]

ARIA-139 Support attributes

* Fully implement attribute support in parser
* New intrinsic function evaluation mechanism
* Implemented more intrinsic functions, including get_attribute
* Fix to one-on-one relationship back population
* Fixes to TOSCA use case examples
* Indirectly related: re-enabled node_filter mechanism and reworked
filter constraints in order to make them serializable
* utils/type is much more robust now and consolidates all conversions
and names
* Moved dsl_specification to new utils/specification (because utils/type
uses it)


> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[4/6] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/definitions.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/definitions.py 
b/extensions/aria_extension_tosca/simple_v1_0/definitions.py
index b60a797..8564249 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/definitions.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/definitions.py
@@ -15,7 +15,7 @@
 
 from aria.utils.collections import FrozenDict
 from aria.utils.caching import cachedmethod
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation import (has_fields, short_form_field, 
allow_unknown_fields,
   primitive_field, primitive_list_field, 
object_field,
   object_list_field, object_dict_field,
@@ -35,7 +35,7 @@ from .modeling.interfaces import 
(get_and_override_input_definitions_from_type,
   
get_and_override_operation_definitions_from_type)
 
 @has_fields
-@dsl_specification('3.5.8', 'tosca-simple-1.0')
+@implements_specification('3.5.8', 'tosca-simple-1.0')
 class PropertyDefinition(ExtensiblePresentation):
 """
 A property definition defines a named, typed value and related data that 
can be associated with
@@ -86,7 +86,7 @@ class PropertyDefinition(ExtensiblePresentation):
 
 @primitive_field(str, default='supported', allowed=('supported', 
'unsupported', 'experimental',
 'deprecated'))
-@dsl_specification(section='3.5.8.3', spec='tosca-simple-1.0')
+@implements_specification(section='3.5.8.3', spec='tosca-simple-1.0')
 def status(self):
 """
 The optional status of the property relative to the specification or 
implementation.
@@ -121,7 +121,7 @@ class PropertyDefinition(ExtensiblePresentation):
 return get_property_constraints(context, self)
 
 @has_fields
-@dsl_specification('3.5.10', 'tosca-simple-1.0')
+@implements_specification('3.5.10', 'tosca-simple-1.0')
 class AttributeDefinition(ExtensiblePresentation):
 """
 An attribute definition defines a named, typed value that can be 
associated with an entity
@@ -190,7 +190,7 @@ class AttributeDefinition(ExtensiblePresentation):
 return get_data_type(context, self, 'type')
 
 @has_fields
-@dsl_specification('3.5.12', 'tosca-simple-1.0')
+@implements_specification('3.5.12', 'tosca-simple-1.0')
 class ParameterDefinition(PropertyDefinition):
 """
 A parameter definition is essentially a TOSCA property definition; 
however, it also allows a
@@ -225,7 +225,7 @@ class ParameterDefinition(PropertyDefinition):
 
 @short_form_field('implementation')
 @has_fields
-@dsl_specification('3.5.13-1', 'tosca-simple-1.0')
+@implements_specification('3.5.13-1', 'tosca-simple-1.0')
 class OperationDefinition(ExtensiblePresentation):
 """
 An operation definition defines a named function or procedure that can be 
bound to an
@@ -266,7 +266,7 @@ class OperationDefinition(ExtensiblePresentation):
 
 @allow_unknown_fields
 @has_fields
-@dsl_specification('3.5.14-1', 'tosca-simple-1.0')
+@implements_specification('3.5.14-1', 'tosca-simple-1.0')
 class InterfaceDefinition(ExtensiblePresentation):
 """
 An interface definition defines a named interface that can be associated 
with a Node or
@@ -352,7 +352,7 @@ class RelationshipDefinition(ExtensiblePresentation):
 
 @short_form_field('capability')
 @has_fields
-@dsl_specification('3.6.2', 'tosca-simple-1.0')
+@implements_specification('3.6.2', 'tosca-simple-1.0')
 class RequirementDefinition(ExtensiblePresentation):
 """
 The Requirement definition describes a named requirement (dependencies) of 
a TOSCA Node Type or
@@ -418,7 +418,7 @@ class RequirementDefinition(ExtensiblePresentation):
 
 @short_form_field('type')
 @has_fields
-@dsl_specification('3.6.1', 'tosca-simple-1.0')
+@implements_specification('3.6.1', 'tosca-simple-1.0')
 class CapabilityDefinition(ExtensiblePresentation):
 """
 A capability definition defines a named, typed set of data that can be 
associated with Node Type

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/filters.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/filters.py 
b/extensions/aria_extension_tosca/simple_v1_0/filters.py
index 617ce7a..838b505 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/filters.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/filters.py
@@ -14,7 +14,7 @@
 # limitations under the License.
 
 from aria.utils.caching import cachedmethod
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation 

[6/6] incubator-ariatosca git commit: ARIA-148 Enhance CLI show commands

[emblemparade]

ARIA-148 Enhance CLI show commands

* Allow "--full" flag to provide a complete dump
* Allow "--json" and "--yaml" flags for dump in those formats
* Support for node graph and type hierarchies
* Some fixes for YAML dump for our custom types
* Also closes ARIA-186: "aria services show" command


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/fdd57c47
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/fdd57c47
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/fdd57c47

Branch: refs/heads/ARIA-148-extra-cli-commands
Commit: fdd57c47acfe63dc25c6f20bcb3785225b3f774d
Parents: 60ea3eb
Author: Tal Liron 
Authored: Thu Apr 20 17:54:47 2017 -0500
Committer: Tal Liron 
Committed: Thu May 11 12:42:40 2017 -0500

--
 aria/cli/commands/service_templates.py |  72 ++
 aria/cli/commands/services.py  |  51 ++-
 aria/cli/core/aria.py  | 203 +++-
 aria/cli/helptexts.py  |  12 +-
 aria/cli/table.py  |   3 +-
 aria/modeling/service_instance.py  |   4 +-
 aria/modeling/service_template.py  |   2 +-
 aria/modeling/types.py |   5 +
 aria/utils/collections.py  |   2 +-
 9 files changed, 260 insertions(+), 94 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/fdd57c47/aria/cli/commands/service_templates.py
--
diff --git a/aria/cli/commands/service_templates.py 
b/aria/cli/commands/service_templates.py
index e459871..0a24907 100644
--- a/aria/cli/commands/service_templates.py
+++ b/aria/cli/commands/service_templates.py
@@ -23,11 +23,13 @@ from .. import utils
 from ..core import aria
 from ...core import Core
 from ...storage import exceptions as storage_exceptions
+from ...parser import consumption
+from ...utils import (formatting, collections, console)
 
 
 DESCRIPTION_FIELD_LENGTH_LIMIT = 20
 SERVICE_TEMPLATE_COLUMNS = \
-['id', 'name', 'description', 'main_file_name', 'created_at', 'updated_at']
+('id', 'name', 'description', 'main_file_name', 'created_at', 'updated_at')
 
 
 @aria.group(name='service-templates')
@@ -43,32 +45,52 @@ def service_templates():
 @aria.argument('service-template-name')
 @aria.options.verbose()
 @aria.pass_model_storage
+@aria.options.service_template_mode_full
+@aria.options.mode_types
+@aria.options.format_json
+@aria.options.format_yaml
 @aria.pass_logger
-def show(service_template_name, model_storage, logger):
-"""Show information for a specific service templates
+def show(service_template_name, model_storage, mode_full, mode_types, 
format_json, format_yaml,
+ logger):
+"""Show information for a specific service template
 
 `SERVICE_TEMPLATE_NAME` is the name of the service template to show 
information on.
 """
-logger.info('Showing service template 
{0}...'.format(service_template_name))
 service_template = 
model_storage.service_template.get_by_name(service_template_name)
-service_template_dict = service_template.to_dict()
-service_template_dict['#services'] = len(service_template.services)
 
-column_formatters = \
-
dict(description=table.trim_formatter_generator(DESCRIPTION_FIELD_LENGTH_LIMIT))
-columns = SERVICE_TEMPLATE_COLUMNS + ['#services']
-table.print_data(columns, service_template_dict, 'Service-template:',
- column_formatters=column_formatters, col_max_width=50)
-
-if service_template_dict['description'] is not None:
-logger.info('Description:')
-
logger.info('{0}{1}'.format(service_template_dict['description'].encode('UTF-8')
 or '',
-os.linesep))
-
-if service_template.services:
-logger.info('Existing services:')
-for service in service_template.services:
-logger.info('\t{0}'.format(service.name))
+if format_json or format_yaml:
+mode_full = True
+
+if mode_full:
+consumption.ConsumptionContext()
+if format_json:
+
console.puts(formatting.json_dumps(collections.prune(service_template.as_raw)))
+elif format_yaml:
+
console.puts(formatting.yaml_dumps(collections.prune(service_template.as_raw)))
+else:
+service_template.dump()
+elif mode_types:
+consumption.ConsumptionContext()
+service_template.dump_types()
+else:
+logger.info('Showing service template 
{0}...'.format(service_template_name))
+service_template_dict = service_template.to_dict()
+service_template_dict['#services'] = len(service_template.services)
+columns = 

[2/6] incubator-ariatosca git commit: ARIA-213 Sporadic tests failures over locked database issue

[emblemparade]

ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/2ee06b8a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/2ee06b8a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/2ee06b8a

Branch: refs/heads/ARIA-148-extra-cli-commands
Commit: 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589
Parents: 6864d42
Author: max-orlov 
Authored: Tue May 9 17:24:31 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 17:24:15 2017 +0300

--
 aria/logger.py  | 25 +
 aria/orchestrator/context/common.py | 13 +--
 .../workflows/core/events_handler.py|  4 +-
 aria/orchestrator/workflows/executor/process.py | 51 ++
 aria/storage/instrumentation.py | 97 +---
 .../orchestrator/workflows/executor/__init__.py | 21 -
 .../workflows/executor/test_executor.py | 36 ++--
 .../workflows/executor/test_process_executor.py | 13 ++-
 tests/storage/test_instrumentation.py   | 37 +++-
 9 files changed, 219 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 97d3878..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -114,17 +114,11 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
-
-return _SQLAlchemyHandler(session=session,
-  engine=engine,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):
@@ -168,10 +162,9 @@ def create_file_log_handler(
 
 class _SQLAlchemyHandler(logging.Handler):
 
-def __init__(self, session, engine, log_cls, execution_id, **kwargs):
+def __init__(self, model, log_cls, execution_id, **kwargs):
 logging.Handler.__init__(self, **kwargs)
-self._session = session
-self._engine = engine
+self._model = model
 self._cls = log_cls
 self._execution_id = execution_id
 
@@ -188,15 +181,7 @@ class _SQLAlchemyHandler(logging.Handler):
 # Not mandatory.
 traceback=getattr(record, 'traceback', None)
 )
-self._session.add(log)
-
-try:
-self._session.commit()
-except BaseException:
-self._session.rollback()
-raise
-finally:
-self._session.close()
+self._model.log.put(log)
 
 
 _default_file_formatter = logging.Formatter(

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/orchestrator/context/common.py
--
diff --git a/aria/orchestrator/context/common.py 
b/aria/orchestrator/context/common.py
index 64ef9a4..0854a27 100644
--- a/aria/orchestrator/context/common.py
+++ b/aria/orchestrator/context/common.py
@@ -79,13 +79,9 @@ class BaseContext(object):
 self.logger.addHandler(self._get_sqla_handler())
 
 def _get_sqla_handler(self):
-api_kwargs = {}
-if self._model._initiator:
-
api_kwargs.update(self._model._initiator(**self._model._initiator_kwargs))
-api_kwargs.update(**self._model._api_kwargs)
-return aria_logger.create_sqla_log_handler(log_cls=modeling.models.Log,
-   
execution_id=self._execution_id,
-   **api_kwargs)
+return aria_logger.create_sqla_log_handler(model=self._model,
+   log_cls=modeling.models.Log,
+   
execution_id=self._execution_id)
 
 def __repr__(self):
 return (
@@ -196,7 

[3/6] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py 
b/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
index f61cb99..9c3ea42 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
@@ -58,7 +58,8 @@ def get_inherited_property_definitions(context, presentation, 
field_name, for_pr
 # NodeTemplate, RelationshipTemplate, GroupTemplate, PolicyTemplate
 #
 
-def get_assigned_and_defined_property_values(context, presentation):
+def get_assigned_and_defined_property_values(context, presentation, 
field_name='property',
+ field_name_plural='properties'):
 """
 Returns the assigned property values while making sure they are defined in 
our type.
 
@@ -70,8 +71,9 @@ def get_assigned_and_defined_property_values(context, 
presentation):
 values = OrderedDict()
 
 the_type = presentation._get_type(context)
-assignments = presentation.properties
-definitions = the_type._get_properties(context) if the_type is not None 
else None
+assignments = getattr(presentation, field_name_plural)
+get_fn_name = '_get_{0}'.format(field_name_plural)
+definitions = getattr(the_type, get_fn_name)(context) if the_type is not 
None else None
 
 # Fill in our assignments, but make sure they are defined
 if assignments:
@@ -80,14 +82,14 @@ def get_assigned_and_defined_property_values(context, 
presentation):
 definition = definitions[name]
 values[name] = coerce_property_value(context, value, 
definition, value.value)
 else:
-context.validation.report('assignment to undefined property 
"%s" in "%s"'
-  % (name, presentation._fullname),
+context.validation.report('assignment to undefined {0} "{1}" 
in "{2}"'
+  .format(field_name, name, 
presentation._fullname),
   locator=value._locator, 
level=Issue.BETWEEN_TYPES)
 
 # Fill in defaults from the definitions
 if definitions:
 for name, definition in definitions.iteritems():
-if (values.get(name) is None) and (definition.default is not None):
+if values.get(name) is None:
 values[name] = coerce_property_value(context, presentation, 
definition,
  definition.default)
 
@@ -181,7 +183,8 @@ def merge_property_definitions(context, presentation, 
property_definitions,
 def coerce_property_value(context, presentation, definition, value, 
aspect=None):
 the_type = definition._get_type(context) if definition is not None else 
None
 entry_schema = definition.entry_schema if definition is not None else None
-constraints = definition._get_constraints(context) if definition is not 
None else None
+constraints = definition._get_constraints(context) \
+if ((definition is not None) and hasattr(definition, 
'_get_constraints')) else None
 value = coerce_value(context, presentation, the_type, entry_schema, 
constraints, value, aspect)
 if (the_type is not None) and hasattr(the_type, '_name'):
 type_name = the_type._name

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
--
diff --git 
a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py 
b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
index 6ff4384..d7b03ae 100644
--- 
a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
+++ 
b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
@@ -16,7 +16,7 @@
 import re
 
 from aria.utils.formatting import safe_repr
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation import (report_issue_for_unknown_type, 
derived_from_validator)
 from aria.parser.validation import Issue
 
@@ -28,8 +28,7 @@ from .types import get_type_by_full_or_shorthand_name, 
convert_shorthand_to_full
 # NodeTemplate, RelationshipTemplate
 #
 
-@dsl_specification('3.7.3.3', 'tosca-simple-1.0')
-@dsl_specification('3.7.4.3', 'tosca-simple-1.0')
+@implements_specification('3.7.3.3', 'tosca-simple-1.0')
 def copy_validator(template_type_name, templates_dict_name):
 """
 Makes sure that the field refers to an existing template defined in the 
root presenter.
@@ -304,8 +303,12 @@ def constraint_clause_pattern_validator(field, 

[jira] [Commented] (ARIA-157) CLI unit tests for storing service-templates fail on appveyor

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006890#comment-16006890
 ] 

ASF subversion and git services commented on ARIA-157:
--

Commit 6864d42e1748b5cd8ecc596fc1b4adc04b0cc9ba in incubator-ariatosca's branch 
refs/heads/ARIA-148-extra-cli-commands from [~avia]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=6864d42 ]

ARIA-157 Failing CLI service-templates store tests on Windows

Three tests from `aria service-templates store` failed on Windows, but
not on Linux.

The reason for this failures was differing implementation of
os.path.dirname across the platforms.

Python implements os.path.dirname in the ntpath module. There, somewhere
down the line of calls, (a part of) the argument of dirname is tested
for membership in a string (using `in`). In these three tests, the
argument of dirname is of type MagicMock, and an error is raised since
only a string can be tested for membership in a string.

The solution was to mock the dirname calls.


> CLI unit tests for storing service-templates fail on appveyor
> -
>
> Key: ARIA-157
> URL: https://issues.apache.org/jira/browse/ARIA-157
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Avia Efrat
>Assignee: Avia Efrat
> Fix For: 0.1.0
>
>
> The failing tests are:
> test_store_no_exception
> test_store_raises_exception_resulting_from_name_uniqueness
> test_store_raises_exception
> They pass on Travis.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[5/6] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

ARIA-139 Support attributes

* Fully implement attribute support in parser
* New intrinsic function evaluation mechanism
* Implemented more intrinsic functions, including get_attribute
* Fix to one-on-one relationship back population
* Fixes to TOSCA use case examples
* Indirectly related: re-enabled node_filter mechanism and reworked
filter constraints in order to make them serializable
* utils/type is much more robust now and consolidates all conversions
and names
* Moved dsl_specification to new utils/specification (because utils/type
uses it)


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/60ea3ebb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/60ea3ebb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/60ea3ebb

Branch: refs/heads/ARIA-148-extra-cli-commands
Commit: 60ea3ebb21e762d36115db26563a93dd3cb72003
Parents: 2ee06b8
Author: Tal Liron 
Authored: Wed Apr 19 20:07:33 2017 -0500
Committer: Tal Liron 
Committed: Thu May 11 11:31:02 2017 -0500

--
 aria/core.py|   6 +-
 aria/modeling/contraints.py |  28 +
 aria/modeling/functions.py  | 104 ++-
 aria/modeling/mixins.py |   2 +-
 aria/modeling/relationship.py   |  19 +-
 aria/modeling/service_common.py | 164 -
 aria/modeling/service_instance.py   |  78 ++-
 aria/modeling/service_template.py   | 126 ++--
 aria/modeling/utils.py  |  50 +-
 .../execution_plugin/instantiation.py   |   2 +-
 aria/parser/__init__.py |   4 +-
 aria/parser/consumption/__init__.py |  11 +-
 aria/parser/consumption/modeling.py |   4 +-
 aria/parser/presentation/fields.py  |   3 +-
 aria/parser/presentation/presentation.py|   3 +-
 aria/parser/presentation/utils.py   |   3 +-
 aria/parser/specification.py|  39 +-
 aria/parser/validation/issue.py |   2 +-
 aria/utils/caching.py   |  13 +-
 aria/utils/formatting.py|  16 +-
 aria/utils/specification.py |  53 ++
 aria/utils/type.py  | 157 -
 .../block-storage-1/block-storage-1.yaml|   2 +-
 .../block-storage-2/block-storage-2.yaml|   2 +-
 .../block-storage-3/block-storage-3.yaml|   2 +-
 .../block-storage-4/block-storage-4.yaml|   2 +-
 .../block-storage-5/block-storage-5.yaml|   6 +-
 .../block-storage-6/block-storage-6.yaml|   4 +-
 .../use-cases/multi-tier-1/multi-tier-1.yaml|  14 +-
 .../simple_v1_0/__init__.py |  12 +-
 .../simple_v1_0/assignments.py  |  16 +-
 .../simple_v1_0/data_types.py   |  20 +-
 .../simple_v1_0/definitions.py  |  18 +-
 .../aria_extension_tosca/simple_v1_0/filters.py |   6 +-
 .../simple_v1_0/functions.py| 536 ---
 .../aria_extension_tosca/simple_v1_0/misc.py|  20 +-
 .../simple_v1_0/modeling/__init__.py| 185 ++---
 .../simple_v1_0/modeling/constraints.py | 144 
 .../simple_v1_0/modeling/data_types.py  |  23 +-
 .../simple_v1_0/modeling/functions.py   | 677 +++
 .../simple_v1_0/modeling/properties.py  |  17 +-
 .../presentation/field_validators.py|  13 +-
 .../simple_v1_0/presenter.py|   4 +-
 .../simple_v1_0/templates.py|  47 +-
 .../aria_extension_tosca/simple_v1_0/types.py   |  18 +-
 .../node-cellar/node-cellar.yaml|   6 +-
 46 files changed, 1644 insertions(+), 1037 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/aria/core.py
--
diff --git a/aria/core.py b/aria/core.py
index af1984a..cc943ef 100644
--- a/aria/core.py
+++ b/aria/core.py
@@ -77,10 +77,14 @@ class Core(object):
 consumption.ConsumerChain(
 context,
 (
+consumption.CoerceServiceInstanceValues,
+consumption.ValidateServiceInstance,
 consumption.SatisfyRequirements,
+consumption.CoerceServiceInstanceValues,
 consumption.ValidateCapabilities,
 consumption.FindHosts,
-consumption.ConfigureOperations
+consumption.ConfigureOperations,
+consumption.CoerceServiceInstanceValues
 )).consume()
   

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006893#comment-16006893
 ] 

ASF subversion and git services commented on ARIA-148:
--

Commit fdd57c47acfe63dc25c6f20bcb3785225b3f774d in incubator-ariatosca's branch 
refs/heads/ARIA-148-extra-cli-commands from [~emblemparade]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=fdd57c4 ]

ARIA-148 Enhance CLI show commands

* Allow "--full" flag to provide a complete dump
* Allow "--json" and "--yaml" flags for dump in those formats
* Support for node graph and type hierarchies
* Some fixes for YAML dump for our custom types
* Also closes ARIA-186: "aria services show" command


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (ARIA-139) TOSCA attributes

[Tal Liron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tal Liron closed ARIA-139.
--
Resolution: Done

> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-139) TOSCA attributes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006759#comment-16006759
 ] 

ASF GitHub Bot commented on ARIA-139:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/118


> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-139) TOSCA attributes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006758#comment-16006758
 ] 

ASF subversion and git services commented on ARIA-139:
--

Commit 60ea3ebb21e762d36115db26563a93dd3cb72003 in incubator-ariatosca's branch 
refs/heads/master from [~emblemparade]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=60ea3eb ]

ARIA-139 Support attributes

* Fully implement attribute support in parser
* New intrinsic function evaluation mechanism
* Implemented more intrinsic functions, including get_attribute
* Fix to one-on-one relationship back population
* Fixes to TOSCA use case examples
* Indirectly related: re-enabled node_filter mechanism and reworked
filter constraints in order to make them serializable
* utils/type is much more robust now and consolidates all conversions
and names
* Moved dsl_specification to new utils/specification (because utils/type
uses it)


> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[3/3] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

ARIA-139 Support attributes

* Fully implement attribute support in parser
* New intrinsic function evaluation mechanism
* Implemented more intrinsic functions, including get_attribute
* Fix to one-on-one relationship back population
* Fixes to TOSCA use case examples
* Indirectly related: re-enabled node_filter mechanism and reworked
filter constraints in order to make them serializable
* utils/type is much more robust now and consolidates all conversions
and names
* Moved dsl_specification to new utils/specification (because utils/type
uses it)


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/60ea3ebb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/60ea3ebb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/60ea3ebb

Branch: refs/heads/master
Commit: 60ea3ebb21e762d36115db26563a93dd3cb72003
Parents: 2ee06b8
Author: Tal Liron 
Authored: Wed Apr 19 20:07:33 2017 -0500
Committer: Tal Liron 
Committed: Thu May 11 11:31:02 2017 -0500

--
 aria/core.py|   6 +-
 aria/modeling/contraints.py |  28 +
 aria/modeling/functions.py  | 104 ++-
 aria/modeling/mixins.py |   2 +-
 aria/modeling/relationship.py   |  19 +-
 aria/modeling/service_common.py | 164 -
 aria/modeling/service_instance.py   |  78 ++-
 aria/modeling/service_template.py   | 126 ++--
 aria/modeling/utils.py  |  50 +-
 .../execution_plugin/instantiation.py   |   2 +-
 aria/parser/__init__.py |   4 +-
 aria/parser/consumption/__init__.py |  11 +-
 aria/parser/consumption/modeling.py |   4 +-
 aria/parser/presentation/fields.py  |   3 +-
 aria/parser/presentation/presentation.py|   3 +-
 aria/parser/presentation/utils.py   |   3 +-
 aria/parser/specification.py|  39 +-
 aria/parser/validation/issue.py |   2 +-
 aria/utils/caching.py   |  13 +-
 aria/utils/formatting.py|  16 +-
 aria/utils/specification.py |  53 ++
 aria/utils/type.py  | 157 -
 .../block-storage-1/block-storage-1.yaml|   2 +-
 .../block-storage-2/block-storage-2.yaml|   2 +-
 .../block-storage-3/block-storage-3.yaml|   2 +-
 .../block-storage-4/block-storage-4.yaml|   2 +-
 .../block-storage-5/block-storage-5.yaml|   6 +-
 .../block-storage-6/block-storage-6.yaml|   4 +-
 .../use-cases/multi-tier-1/multi-tier-1.yaml|  14 +-
 .../simple_v1_0/__init__.py |  12 +-
 .../simple_v1_0/assignments.py  |  16 +-
 .../simple_v1_0/data_types.py   |  20 +-
 .../simple_v1_0/definitions.py  |  18 +-
 .../aria_extension_tosca/simple_v1_0/filters.py |   6 +-
 .../simple_v1_0/functions.py| 536 ---
 .../aria_extension_tosca/simple_v1_0/misc.py|  20 +-
 .../simple_v1_0/modeling/__init__.py| 185 ++---
 .../simple_v1_0/modeling/constraints.py | 144 
 .../simple_v1_0/modeling/data_types.py  |  23 +-
 .../simple_v1_0/modeling/functions.py   | 677 +++
 .../simple_v1_0/modeling/properties.py  |  17 +-
 .../presentation/field_validators.py|  13 +-
 .../simple_v1_0/presenter.py|   4 +-
 .../simple_v1_0/templates.py|  47 +-
 .../aria_extension_tosca/simple_v1_0/types.py   |  18 +-
 .../node-cellar/node-cellar.yaml|   6 +-
 46 files changed, 1644 insertions(+), 1037 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/aria/core.py
--
diff --git a/aria/core.py b/aria/core.py
index af1984a..cc943ef 100644
--- a/aria/core.py
+++ b/aria/core.py
@@ -77,10 +77,14 @@ class Core(object):
 consumption.ConsumerChain(
 context,
 (
+consumption.CoerceServiceInstanceValues,
+consumption.ValidateServiceInstance,
 consumption.SatisfyRequirements,
+consumption.CoerceServiceInstanceValues,
 consumption.ValidateCapabilities,
 consumption.FindHosts,
-consumption.ConfigureOperations
+consumption.ConfigureOperations,
+consumption.CoerceServiceInstanceValues
 )).consume()
 if 

[2/3] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/definitions.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/definitions.py 
b/extensions/aria_extension_tosca/simple_v1_0/definitions.py
index b60a797..8564249 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/definitions.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/definitions.py
@@ -15,7 +15,7 @@
 
 from aria.utils.collections import FrozenDict
 from aria.utils.caching import cachedmethod
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation import (has_fields, short_form_field, 
allow_unknown_fields,
   primitive_field, primitive_list_field, 
object_field,
   object_list_field, object_dict_field,
@@ -35,7 +35,7 @@ from .modeling.interfaces import 
(get_and_override_input_definitions_from_type,
   
get_and_override_operation_definitions_from_type)
 
 @has_fields
-@dsl_specification('3.5.8', 'tosca-simple-1.0')
+@implements_specification('3.5.8', 'tosca-simple-1.0')
 class PropertyDefinition(ExtensiblePresentation):
 """
 A property definition defines a named, typed value and related data that 
can be associated with
@@ -86,7 +86,7 @@ class PropertyDefinition(ExtensiblePresentation):
 
 @primitive_field(str, default='supported', allowed=('supported', 
'unsupported', 'experimental',
 'deprecated'))
-@dsl_specification(section='3.5.8.3', spec='tosca-simple-1.0')
+@implements_specification(section='3.5.8.3', spec='tosca-simple-1.0')
 def status(self):
 """
 The optional status of the property relative to the specification or 
implementation.
@@ -121,7 +121,7 @@ class PropertyDefinition(ExtensiblePresentation):
 return get_property_constraints(context, self)
 
 @has_fields
-@dsl_specification('3.5.10', 'tosca-simple-1.0')
+@implements_specification('3.5.10', 'tosca-simple-1.0')
 class AttributeDefinition(ExtensiblePresentation):
 """
 An attribute definition defines a named, typed value that can be 
associated with an entity
@@ -190,7 +190,7 @@ class AttributeDefinition(ExtensiblePresentation):
 return get_data_type(context, self, 'type')
 
 @has_fields
-@dsl_specification('3.5.12', 'tosca-simple-1.0')
+@implements_specification('3.5.12', 'tosca-simple-1.0')
 class ParameterDefinition(PropertyDefinition):
 """
 A parameter definition is essentially a TOSCA property definition; 
however, it also allows a
@@ -225,7 +225,7 @@ class ParameterDefinition(PropertyDefinition):
 
 @short_form_field('implementation')
 @has_fields
-@dsl_specification('3.5.13-1', 'tosca-simple-1.0')
+@implements_specification('3.5.13-1', 'tosca-simple-1.0')
 class OperationDefinition(ExtensiblePresentation):
 """
 An operation definition defines a named function or procedure that can be 
bound to an
@@ -266,7 +266,7 @@ class OperationDefinition(ExtensiblePresentation):
 
 @allow_unknown_fields
 @has_fields
-@dsl_specification('3.5.14-1', 'tosca-simple-1.0')
+@implements_specification('3.5.14-1', 'tosca-simple-1.0')
 class InterfaceDefinition(ExtensiblePresentation):
 """
 An interface definition defines a named interface that can be associated 
with a Node or
@@ -352,7 +352,7 @@ class RelationshipDefinition(ExtensiblePresentation):
 
 @short_form_field('capability')
 @has_fields
-@dsl_specification('3.6.2', 'tosca-simple-1.0')
+@implements_specification('3.6.2', 'tosca-simple-1.0')
 class RequirementDefinition(ExtensiblePresentation):
 """
 The Requirement definition describes a named requirement (dependencies) of 
a TOSCA Node Type or
@@ -418,7 +418,7 @@ class RequirementDefinition(ExtensiblePresentation):
 
 @short_form_field('type')
 @has_fields
-@dsl_specification('3.6.1', 'tosca-simple-1.0')
+@implements_specification('3.6.1', 'tosca-simple-1.0')
 class CapabilityDefinition(ExtensiblePresentation):
 """
 A capability definition defines a named, typed set of data that can be 
associated with Node Type

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/filters.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/filters.py 
b/extensions/aria_extension_tosca/simple_v1_0/filters.py
index 617ce7a..838b505 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/filters.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/filters.py
@@ -14,7 +14,7 @@
 # limitations under the License.
 
 from aria.utils.caching import cachedmethod
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation 

[GitHub] incubator-ariatosca pull request #118: ARIA-139 Support attributes

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/118


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/3] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/master 2ee06b8a6 -> 60ea3ebb2


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py 
b/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
index f61cb99..9c3ea42 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
@@ -58,7 +58,8 @@ def get_inherited_property_definitions(context, presentation, 
field_name, for_pr
 # NodeTemplate, RelationshipTemplate, GroupTemplate, PolicyTemplate
 #
 
-def get_assigned_and_defined_property_values(context, presentation):
+def get_assigned_and_defined_property_values(context, presentation, 
field_name='property',
+ field_name_plural='properties'):
 """
 Returns the assigned property values while making sure they are defined in 
our type.
 
@@ -70,8 +71,9 @@ def get_assigned_and_defined_property_values(context, 
presentation):
 values = OrderedDict()
 
 the_type = presentation._get_type(context)
-assignments = presentation.properties
-definitions = the_type._get_properties(context) if the_type is not None 
else None
+assignments = getattr(presentation, field_name_plural)
+get_fn_name = '_get_{0}'.format(field_name_plural)
+definitions = getattr(the_type, get_fn_name)(context) if the_type is not 
None else None
 
 # Fill in our assignments, but make sure they are defined
 if assignments:
@@ -80,14 +82,14 @@ def get_assigned_and_defined_property_values(context, 
presentation):
 definition = definitions[name]
 values[name] = coerce_property_value(context, value, 
definition, value.value)
 else:
-context.validation.report('assignment to undefined property 
"%s" in "%s"'
-  % (name, presentation._fullname),
+context.validation.report('assignment to undefined {0} "{1}" 
in "{2}"'
+  .format(field_name, name, 
presentation._fullname),
   locator=value._locator, 
level=Issue.BETWEEN_TYPES)
 
 # Fill in defaults from the definitions
 if definitions:
 for name, definition in definitions.iteritems():
-if (values.get(name) is None) and (definition.default is not None):
+if values.get(name) is None:
 values[name] = coerce_property_value(context, presentation, 
definition,
  definition.default)
 
@@ -181,7 +183,8 @@ def merge_property_definitions(context, presentation, 
property_definitions,
 def coerce_property_value(context, presentation, definition, value, 
aspect=None):
 the_type = definition._get_type(context) if definition is not None else 
None
 entry_schema = definition.entry_schema if definition is not None else None
-constraints = definition._get_constraints(context) if definition is not 
None else None
+constraints = definition._get_constraints(context) \
+if ((definition is not None) and hasattr(definition, 
'_get_constraints')) else None
 value = coerce_value(context, presentation, the_type, entry_schema, 
constraints, value, aspect)
 if (the_type is not None) and hasattr(the_type, '_name'):
 type_name = the_type._name

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
--
diff --git 
a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py 
b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
index 6ff4384..d7b03ae 100644
--- 
a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
+++ 
b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
@@ -16,7 +16,7 @@
 import re
 
 from aria.utils.formatting import safe_repr
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation import (report_issue_for_unknown_type, 
derived_from_validator)
 from aria.parser.validation import Issue
 
@@ -28,8 +28,7 @@ from .types import get_type_by_full_or_shorthand_name, 
convert_shorthand_to_full
 # NodeTemplate, RelationshipTemplate
 #
 
-@dsl_specification('3.7.3.3', 'tosca-simple-1.0')
-@dsl_specification('3.7.4.3', 'tosca-simple-1.0')
+@implements_specification('3.7.3.3', 'tosca-simple-1.0')
 def copy_validator(template_type_name, templates_dict_name):
 """
 Makes sure that the field refers to an existing template defined in the 

[2/6] incubator-ariatosca git commit: ARIA-157 Failing CLI service-templates store tests on Windows

[emblemparade]

ARIA-157 Failing CLI service-templates store tests on Windows

Three tests from `aria service-templates store` failed on Windows, but
not on Linux.

The reason for this failures was differing implementation of
os.path.dirname across the platforms.

Python implements os.path.dirname in the ntpath module. There, somewhere
down the line of calls, (a part of) the argument of dirname is tested
for membership in a string (using `in`). In these three tests, the
argument of dirname is of type MagicMock, and an error is raised since
only a string can be tested for membership in a string.

The solution was to mock the dirname calls.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/6864d42e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/6864d42e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/6864d42e

Branch: refs/heads/ARIA-139-attributes
Commit: 6864d42e1748b5cd8ecc596fc1b4adc04b0cc9ba
Parents: 0af9e63
Author: Avia Efrat 
Authored: Wed May 10 17:39:49 2017 +0300
Committer: Avia Efrat 
Committed: Wed May 10 18:13:02 2017 +0300

--
 tests/cli/test_service_templates.py | 3 +++
 1 file changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/6864d42e/tests/cli/test_service_templates.py
--
diff --git a/tests/cli/test_service_templates.py 
b/tests/cli/test_service_templates.py
index 22a8fc8..bc3c751 100644
--- a/tests/cli/test_service_templates.py
+++ b/tests/cli/test_service_templates.py
@@ -128,6 +128,7 @@ class TestServiceTemplatesStore(TestCliBase):
 
 monkeypatch.setattr(Core, 'create_service_template', mock_object)
 monkeypatch.setattr(service_template_utils, 'get', mock_object)
+monkeypatch.setattr(os.path, 'dirname', mock_object)
 self.invoke('service_templates store stubpath {name}'.format(
 name=mock_models.SERVICE_TEMPLATE_NAME))
 assert 'Service template {name} stored'.format(
@@ -152,6 +153,7 @@ class TestServiceTemplatesStore(TestCliBase):
 'create_service_template',
 raise_exception(storage_exceptions.NotFoundError,
 msg='UNIQUE constraint failed'))
+monkeypatch.setattr(os.path, 'dirname', mock_object)
 
 assert_exception_raised(
 self.invoke('service_templates store stubpath test_st'),
@@ -164,6 +166,7 @@ class TestServiceTemplatesStore(TestCliBase):
 monkeypatch.setattr(Core,
 'create_service_template',
 raise_exception(storage_exceptions.NotFoundError))
+monkeypatch.setattr(os.path, 'dirname', mock_object)
 
 assert_exception_raised(
 self.invoke('service_templates store stubpath test_st'),

[jira] [Commented] (ARIA-157) CLI unit tests for storing service-templates fail on appveyor

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006722#comment-16006722
 ] 

ASF subversion and git services commented on ARIA-157:
--

Commit 6864d42e1748b5cd8ecc596fc1b4adc04b0cc9ba in incubator-ariatosca's branch 
refs/heads/ARIA-139-attributes from [~avia]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=6864d42 ]

ARIA-157 Failing CLI service-templates store tests on Windows

Three tests from `aria service-templates store` failed on Windows, but
not on Linux.

The reason for this failures was differing implementation of
os.path.dirname across the platforms.

Python implements os.path.dirname in the ntpath module. There, somewhere
down the line of calls, (a part of) the argument of dirname is tested
for membership in a string (using `in`). In these three tests, the
argument of dirname is of type MagicMock, and an error is raised since
only a string can be tested for membership in a string.

The solution was to mock the dirname calls.


> CLI unit tests for storing service-templates fail on appveyor
> -
>
> Key: ARIA-157
> URL: https://issues.apache.org/jira/browse/ARIA-157
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Avia Efrat
>Assignee: Avia Efrat
> Fix For: 0.1.0
>
>
> The failing tests are:
> test_store_no_exception
> test_store_raises_exception_resulting_from_name_uniqueness
> test_store_raises_exception
> They pass on Travis.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[3/6] incubator-ariatosca git commit: ARIA-213 Sporadic tests failures over locked database issue

[emblemparade]

ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/2ee06b8a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/2ee06b8a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/2ee06b8a

Branch: refs/heads/ARIA-139-attributes
Commit: 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589
Parents: 6864d42
Author: max-orlov 
Authored: Tue May 9 17:24:31 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 17:24:15 2017 +0300

--
 aria/logger.py  | 25 +
 aria/orchestrator/context/common.py | 13 +--
 .../workflows/core/events_handler.py|  4 +-
 aria/orchestrator/workflows/executor/process.py | 51 ++
 aria/storage/instrumentation.py | 97 +---
 .../orchestrator/workflows/executor/__init__.py | 21 -
 .../workflows/executor/test_executor.py | 36 ++--
 .../workflows/executor/test_process_executor.py | 13 ++-
 tests/storage/test_instrumentation.py   | 37 +++-
 9 files changed, 219 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 97d3878..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -114,17 +114,11 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
-
-return _SQLAlchemyHandler(session=session,
-  engine=engine,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):
@@ -168,10 +162,9 @@ def create_file_log_handler(
 
 class _SQLAlchemyHandler(logging.Handler):
 
-def __init__(self, session, engine, log_cls, execution_id, **kwargs):
+def __init__(self, model, log_cls, execution_id, **kwargs):
 logging.Handler.__init__(self, **kwargs)
-self._session = session
-self._engine = engine
+self._model = model
 self._cls = log_cls
 self._execution_id = execution_id
 
@@ -188,15 +181,7 @@ class _SQLAlchemyHandler(logging.Handler):
 # Not mandatory.
 traceback=getattr(record, 'traceback', None)
 )
-self._session.add(log)
-
-try:
-self._session.commit()
-except BaseException:
-self._session.rollback()
-raise
-finally:
-self._session.close()
+self._model.log.put(log)
 
 
 _default_file_formatter = logging.Formatter(

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/orchestrator/context/common.py
--
diff --git a/aria/orchestrator/context/common.py 
b/aria/orchestrator/context/common.py
index 64ef9a4..0854a27 100644
--- a/aria/orchestrator/context/common.py
+++ b/aria/orchestrator/context/common.py
@@ -79,13 +79,9 @@ class BaseContext(object):
 self.logger.addHandler(self._get_sqla_handler())
 
 def _get_sqla_handler(self):
-api_kwargs = {}
-if self._model._initiator:
-
api_kwargs.update(self._model._initiator(**self._model._initiator_kwargs))
-api_kwargs.update(**self._model._api_kwargs)
-return aria_logger.create_sqla_log_handler(log_cls=modeling.models.Log,
-   
execution_id=self._execution_id,
-   **api_kwargs)
+return aria_logger.create_sqla_log_handler(model=self._model,
+   log_cls=modeling.models.Log,
+   
execution_id=self._execution_id)
 
 def __repr__(self):
 return (
@@ -196,7 +192,6 @@ 

[jira] [Commented] (ARIA-139) TOSCA attributes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006725#comment-16006725
 ] 

ASF subversion and git services commented on ARIA-139:
--

Commit 60ea3ebb21e762d36115db26563a93dd3cb72003 in incubator-ariatosca's branch 
refs/heads/ARIA-139-attributes from [~emblemparade]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=60ea3eb ]

ARIA-139 Support attributes

* Fully implement attribute support in parser
* New intrinsic function evaluation mechanism
* Implemented more intrinsic functions, including get_attribute
* Fix to one-on-one relationship back population
* Fixes to TOSCA use case examples
* Indirectly related: re-enabled node_filter mechanism and reworked
filter constraints in order to make them serializable
* utils/type is much more robust now and consolidates all conversions
and names
* Moved dsl_specification to new utils/specification (because utils/type
uses it)


> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006723#comment-16006723
 ] 

ASF subversion and git services commented on ARIA-213:
--

Commit 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589 in incubator-ariatosca's branch 
refs/heads/ARIA-139-attributes from [~Mxmrlv]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=2ee06b8 ]

ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def 

[1/6] incubator-ariatosca git commit: ARIA-62 Apply pylint on extensions [Forced Update!]

[emblemparade]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-139-attributes fc16f6e00 -> 60ea3ebb2 (forced update)


ARIA-62 Apply pylint on extensions


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/0af9e638
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/0af9e638
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/0af9e638

Branch: refs/heads/ARIA-139-attributes
Commit: 0af9e638b7c75eea6eac39374b7176f9dc2c645a
Parents: eae44d0
Author: Ran Ziv 
Authored: Wed May 10 12:08:01 2017 +0300
Committer: Ran Ziv 
Committed: Wed May 10 12:08:01 2017 +0300

--
 aria/.pylintrc|  2 +-
 .../aria_extension_tosca/simple_nfv_v1_0/presenter.py |  2 +-
 .../aria_extension_tosca/simple_v1_0/assignments.py   |  2 +-
 extensions/aria_extension_tosca/simple_v1_0/functions.py  |  8 
 .../aria_extension_tosca/simple_v1_0/modeling/__init__.py | 10 ++
 extensions/aria_extension_tosca/simple_v1_0/presenter.py  |  2 +-
 tests/.pylintrc   |  2 +-
 tox.ini   |  2 +-
 8 files changed, 16 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0af9e638/aria/.pylintrc
--
diff --git a/aria/.pylintrc b/aria/.pylintrc
index 7da8c56..4d77556 100644
--- a/aria/.pylintrc
+++ b/aria/.pylintrc
@@ -360,7 +360,7 @@ max-locals=20
 max-returns=10
 
 # Maximum number of branch for function / method body
-max-branches=12
+max-branches=15
 
 # Maximum number of statements in function / method body
 max-statements=50

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0af9e638/extensions/aria_extension_tosca/simple_nfv_v1_0/presenter.py
--
diff --git a/extensions/aria_extension_tosca/simple_nfv_v1_0/presenter.py 
b/extensions/aria_extension_tosca/simple_nfv_v1_0/presenter.py
index 0ce918e..cd07f42 100644
--- a/extensions/aria_extension_tosca/simple_nfv_v1_0/presenter.py
+++ b/extensions/aria_extension_tosca/simple_nfv_v1_0/presenter.py
@@ -19,7 +19,7 @@ from aria.utils.caching import cachedmethod
 from ..simple_v1_0 import ToscaSimplePresenter1_0
 
 
-class ToscaSimpleNfvPresenter1_0(ToscaSimplePresenter1_0): # pylint: 
disable=invalid-name
+class ToscaSimpleNfvPresenter1_0(ToscaSimplePresenter1_0): # pylint: 
disable=invalid-name,abstract-method
 """
 ARIA presenter for the `TOSCA Simple Profile for NFV v1.0 csd03 
`__.

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0af9e638/extensions/aria_extension_tosca/simple_v1_0/assignments.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/assignments.py 
b/extensions/aria_extension_tosca/simple_v1_0/assignments.py
index 6e36ba8..9a2179a 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/assignments.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/assignments.py
@@ -94,7 +94,7 @@ class OperationAssignment(ExtensiblePresentation):
 if operation_definition is not None:
 if operation_definition._extensions:
 extensions.update(operation_definition._extensions)
-
+
 extensions = {}
 update_inherited_extensions(extensions, 
self._container._get_type(context))
 if self._container._extensions:

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0af9e638/extensions/aria_extension_tosca/simple_v1_0/functions.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/functions.py 
b/extensions/aria_extension_tosca/simple_v1_0/functions.py
index 405aa8f..2f77420 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/functions.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/functions.py
@@ -176,7 +176,7 @@ class GetProperty(Function):
 
 for modelable_entity in modelable_entities:
 properties = None
-
+
 if hasattr(modelable_entity, 'requirement_templates') \
 and modelable_entity.requirement_templates \
 and (req_or_cap_name in [v.name for v in 
modelable_entity.requirement_templates]):
@@ -185,7 +185,7 @@ class GetProperty(Function):
 # First argument refers to a requirement
 # TODO: should follow to matched capability in other 
node...
 raise 

[5/6] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/definitions.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/definitions.py 
b/extensions/aria_extension_tosca/simple_v1_0/definitions.py
index b60a797..8564249 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/definitions.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/definitions.py
@@ -15,7 +15,7 @@
 
 from aria.utils.collections import FrozenDict
 from aria.utils.caching import cachedmethod
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation import (has_fields, short_form_field, 
allow_unknown_fields,
   primitive_field, primitive_list_field, 
object_field,
   object_list_field, object_dict_field,
@@ -35,7 +35,7 @@ from .modeling.interfaces import 
(get_and_override_input_definitions_from_type,
   
get_and_override_operation_definitions_from_type)
 
 @has_fields
-@dsl_specification('3.5.8', 'tosca-simple-1.0')
+@implements_specification('3.5.8', 'tosca-simple-1.0')
 class PropertyDefinition(ExtensiblePresentation):
 """
 A property definition defines a named, typed value and related data that 
can be associated with
@@ -86,7 +86,7 @@ class PropertyDefinition(ExtensiblePresentation):
 
 @primitive_field(str, default='supported', allowed=('supported', 
'unsupported', 'experimental',
 'deprecated'))
-@dsl_specification(section='3.5.8.3', spec='tosca-simple-1.0')
+@implements_specification(section='3.5.8.3', spec='tosca-simple-1.0')
 def status(self):
 """
 The optional status of the property relative to the specification or 
implementation.
@@ -121,7 +121,7 @@ class PropertyDefinition(ExtensiblePresentation):
 return get_property_constraints(context, self)
 
 @has_fields
-@dsl_specification('3.5.10', 'tosca-simple-1.0')
+@implements_specification('3.5.10', 'tosca-simple-1.0')
 class AttributeDefinition(ExtensiblePresentation):
 """
 An attribute definition defines a named, typed value that can be 
associated with an entity
@@ -190,7 +190,7 @@ class AttributeDefinition(ExtensiblePresentation):
 return get_data_type(context, self, 'type')
 
 @has_fields
-@dsl_specification('3.5.12', 'tosca-simple-1.0')
+@implements_specification('3.5.12', 'tosca-simple-1.0')
 class ParameterDefinition(PropertyDefinition):
 """
 A parameter definition is essentially a TOSCA property definition; 
however, it also allows a
@@ -225,7 +225,7 @@ class ParameterDefinition(PropertyDefinition):
 
 @short_form_field('implementation')
 @has_fields
-@dsl_specification('3.5.13-1', 'tosca-simple-1.0')
+@implements_specification('3.5.13-1', 'tosca-simple-1.0')
 class OperationDefinition(ExtensiblePresentation):
 """
 An operation definition defines a named function or procedure that can be 
bound to an
@@ -266,7 +266,7 @@ class OperationDefinition(ExtensiblePresentation):
 
 @allow_unknown_fields
 @has_fields
-@dsl_specification('3.5.14-1', 'tosca-simple-1.0')
+@implements_specification('3.5.14-1', 'tosca-simple-1.0')
 class InterfaceDefinition(ExtensiblePresentation):
 """
 An interface definition defines a named interface that can be associated 
with a Node or
@@ -352,7 +352,7 @@ class RelationshipDefinition(ExtensiblePresentation):
 
 @short_form_field('capability')
 @has_fields
-@dsl_specification('3.6.2', 'tosca-simple-1.0')
+@implements_specification('3.6.2', 'tosca-simple-1.0')
 class RequirementDefinition(ExtensiblePresentation):
 """
 The Requirement definition describes a named requirement (dependencies) of 
a TOSCA Node Type or
@@ -418,7 +418,7 @@ class RequirementDefinition(ExtensiblePresentation):
 
 @short_form_field('type')
 @has_fields
-@dsl_specification('3.6.1', 'tosca-simple-1.0')
+@implements_specification('3.6.1', 'tosca-simple-1.0')
 class CapabilityDefinition(ExtensiblePresentation):
 """
 A capability definition defines a named, typed set of data that can be 
associated with Node Type

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/filters.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/filters.py 
b/extensions/aria_extension_tosca/simple_v1_0/filters.py
index 617ce7a..838b505 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/filters.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/filters.py
@@ -14,7 +14,7 @@
 # limitations under the License.
 
 from aria.utils.caching import cachedmethod
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation 

[6/6] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

ARIA-139 Support attributes

* Fully implement attribute support in parser
* New intrinsic function evaluation mechanism
* Implemented more intrinsic functions, including get_attribute
* Fix to one-on-one relationship back population
* Fixes to TOSCA use case examples
* Indirectly related: re-enabled node_filter mechanism and reworked
filter constraints in order to make them serializable
* utils/type is much more robust now and consolidates all conversions
and names
* Moved dsl_specification to new utils/specification (because utils/type
uses it)


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/60ea3ebb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/60ea3ebb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/60ea3ebb

Branch: refs/heads/ARIA-139-attributes
Commit: 60ea3ebb21e762d36115db26563a93dd3cb72003
Parents: 2ee06b8
Author: Tal Liron 
Authored: Wed Apr 19 20:07:33 2017 -0500
Committer: Tal Liron 
Committed: Thu May 11 11:31:02 2017 -0500

--
 aria/core.py|   6 +-
 aria/modeling/contraints.py |  28 +
 aria/modeling/functions.py  | 104 ++-
 aria/modeling/mixins.py |   2 +-
 aria/modeling/relationship.py   |  19 +-
 aria/modeling/service_common.py | 164 -
 aria/modeling/service_instance.py   |  78 ++-
 aria/modeling/service_template.py   | 126 ++--
 aria/modeling/utils.py  |  50 +-
 .../execution_plugin/instantiation.py   |   2 +-
 aria/parser/__init__.py |   4 +-
 aria/parser/consumption/__init__.py |  11 +-
 aria/parser/consumption/modeling.py |   4 +-
 aria/parser/presentation/fields.py  |   3 +-
 aria/parser/presentation/presentation.py|   3 +-
 aria/parser/presentation/utils.py   |   3 +-
 aria/parser/specification.py|  39 +-
 aria/parser/validation/issue.py |   2 +-
 aria/utils/caching.py   |  13 +-
 aria/utils/formatting.py|  16 +-
 aria/utils/specification.py |  53 ++
 aria/utils/type.py  | 157 -
 .../block-storage-1/block-storage-1.yaml|   2 +-
 .../block-storage-2/block-storage-2.yaml|   2 +-
 .../block-storage-3/block-storage-3.yaml|   2 +-
 .../block-storage-4/block-storage-4.yaml|   2 +-
 .../block-storage-5/block-storage-5.yaml|   6 +-
 .../block-storage-6/block-storage-6.yaml|   4 +-
 .../use-cases/multi-tier-1/multi-tier-1.yaml|  14 +-
 .../simple_v1_0/__init__.py |  12 +-
 .../simple_v1_0/assignments.py  |  16 +-
 .../simple_v1_0/data_types.py   |  20 +-
 .../simple_v1_0/definitions.py  |  18 +-
 .../aria_extension_tosca/simple_v1_0/filters.py |   6 +-
 .../simple_v1_0/functions.py| 536 ---
 .../aria_extension_tosca/simple_v1_0/misc.py|  20 +-
 .../simple_v1_0/modeling/__init__.py| 185 ++---
 .../simple_v1_0/modeling/constraints.py | 144 
 .../simple_v1_0/modeling/data_types.py  |  23 +-
 .../simple_v1_0/modeling/functions.py   | 677 +++
 .../simple_v1_0/modeling/properties.py  |  17 +-
 .../presentation/field_validators.py|  13 +-
 .../simple_v1_0/presenter.py|   4 +-
 .../simple_v1_0/templates.py|  47 +-
 .../aria_extension_tosca/simple_v1_0/types.py   |  18 +-
 .../node-cellar/node-cellar.yaml|   6 +-
 46 files changed, 1644 insertions(+), 1037 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/aria/core.py
--
diff --git a/aria/core.py b/aria/core.py
index af1984a..cc943ef 100644
--- a/aria/core.py
+++ b/aria/core.py
@@ -77,10 +77,14 @@ class Core(object):
 consumption.ConsumerChain(
 context,
 (
+consumption.CoerceServiceInstanceValues,
+consumption.ValidateServiceInstance,
 consumption.SatisfyRequirements,
+consumption.CoerceServiceInstanceValues,
 consumption.ValidateCapabilities,
 consumption.FindHosts,
-consumption.ConfigureOperations
+consumption.ConfigureOperations,
+consumption.CoerceServiceInstanceValues
 )).consume()
 if 

[jira] [Commented] (ARIA-62) PyLint should cover "extensions", not just "aria"

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006721#comment-16006721
 ] 

ASF subversion and git services commented on ARIA-62:
-

Commit 0af9e638b7c75eea6eac39374b7176f9dc2c645a in incubator-ariatosca's branch 
refs/heads/ARIA-139-attributes from [~ran]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=0af9e63 ]

ARIA-62 Apply pylint on extensions


> PyLint should cover "extensions", not just "aria"
> -
>
> Key: ARIA-62
> URL: https://issues.apache.org/jira/browse/ARIA-62
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Ran Ziv
> Fix For: 0.1.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[4/6] incubator-ariatosca git commit: ARIA-139 Support attributes

[emblemparade]

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
--
diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py 
b/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
index f61cb99..9c3ea42 100644
--- a/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
+++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/properties.py
@@ -58,7 +58,8 @@ def get_inherited_property_definitions(context, presentation, 
field_name, for_pr
 # NodeTemplate, RelationshipTemplate, GroupTemplate, PolicyTemplate
 #
 
-def get_assigned_and_defined_property_values(context, presentation):
+def get_assigned_and_defined_property_values(context, presentation, 
field_name='property',
+ field_name_plural='properties'):
 """
 Returns the assigned property values while making sure they are defined in 
our type.
 
@@ -70,8 +71,9 @@ def get_assigned_and_defined_property_values(context, 
presentation):
 values = OrderedDict()
 
 the_type = presentation._get_type(context)
-assignments = presentation.properties
-definitions = the_type._get_properties(context) if the_type is not None 
else None
+assignments = getattr(presentation, field_name_plural)
+get_fn_name = '_get_{0}'.format(field_name_plural)
+definitions = getattr(the_type, get_fn_name)(context) if the_type is not 
None else None
 
 # Fill in our assignments, but make sure they are defined
 if assignments:
@@ -80,14 +82,14 @@ def get_assigned_and_defined_property_values(context, 
presentation):
 definition = definitions[name]
 values[name] = coerce_property_value(context, value, 
definition, value.value)
 else:
-context.validation.report('assignment to undefined property 
"%s" in "%s"'
-  % (name, presentation._fullname),
+context.validation.report('assignment to undefined {0} "{1}" 
in "{2}"'
+  .format(field_name, name, 
presentation._fullname),
   locator=value._locator, 
level=Issue.BETWEEN_TYPES)
 
 # Fill in defaults from the definitions
 if definitions:
 for name, definition in definitions.iteritems():
-if (values.get(name) is None) and (definition.default is not None):
+if values.get(name) is None:
 values[name] = coerce_property_value(context, presentation, 
definition,
  definition.default)
 
@@ -181,7 +183,8 @@ def merge_property_definitions(context, presentation, 
property_definitions,
 def coerce_property_value(context, presentation, definition, value, 
aspect=None):
 the_type = definition._get_type(context) if definition is not None else 
None
 entry_schema = definition.entry_schema if definition is not None else None
-constraints = definition._get_constraints(context) if definition is not 
None else None
+constraints = definition._get_constraints(context) \
+if ((definition is not None) and hasattr(definition, 
'_get_constraints')) else None
 value = coerce_value(context, presentation, the_type, entry_schema, 
constraints, value, aspect)
 if (the_type is not None) and hasattr(the_type, '_name'):
 type_name = the_type._name

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/60ea3ebb/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
--
diff --git 
a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py 
b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
index 6ff4384..d7b03ae 100644
--- 
a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
+++ 
b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py
@@ -16,7 +16,7 @@
 import re
 
 from aria.utils.formatting import safe_repr
-from aria.parser import dsl_specification
+from aria.parser import implements_specification
 from aria.parser.presentation import (report_issue_for_unknown_type, 
derived_from_validator)
 from aria.parser.validation import Issue
 
@@ -28,8 +28,7 @@ from .types import get_type_by_full_or_shorthand_name, 
convert_shorthand_to_full
 # NodeTemplate, RelationshipTemplate
 #
 
-@dsl_specification('3.7.3.3', 'tosca-simple-1.0')
-@dsl_specification('3.7.4.3', 'tosca-simple-1.0')
+@implements_specification('3.7.3.3', 'tosca-simple-1.0')
 def copy_validator(template_type_name, templates_dict_name):
 """
 Makes sure that the field refers to an existing template defined in the 
root presenter.
@@ -304,8 +303,12 @@ def constraint_clause_pattern_validator(field, 

[jira] [Updated] (ARIA-249) Auto-install plugins wagons from service-template archives

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIA-249:
-
Summary: Auto-install plugins wagons from service-template archives  (was: 
Install plugins )

> Auto-install plugins wagons from service-template archives
> --
>
> Key: ARIA-249
> URL: https://issues.apache.org/jira/browse/ARIA-249
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Priority: Trivial
>
> ARIA supports a single path for installing plugins: By using the 
> {{PluginManager}} class to install wagon archives. A user can use the CLI to 
> install a plugin this way.
> It might be useful if it were also possible to automatically install plugin 
> wagons that are inside a service-template archive (or possibly only to do so 
> if a specific flag is passed when creating a service).
> This could simplify the sharing of service-template archives in between users 
> with different ARIA installations, as all the service-template resources, 
> including plugins which are required by the service-template, will be 
> packaged into one archive. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ARIA-249) Install plugins

[Ran Ziv (JIRA)]

Ran Ziv created ARIA-249:


 Summary: Install plugins 
 Key: ARIA-249
 URL: https://issues.apache.org/jira/browse/ARIA-249
 Project: AriaTosca
  Issue Type: Story
Reporter: Ran Ziv
Priority: Trivial


ARIA supports a single path for installing plugins: By using the 
{{PluginManager}} class to install wagon archives. A user can use the CLI to 
install a plugin this way.

It might be useful if it were also possible to automatically install plugin 
wagons that are inside a service-template archive (or possibly only to do so if 
a specific flag is passed when creating a service).
This could simplify the sharing of service-template archives in between users 
with different ARIA installations, as all the service-template resources, 
including plugins which are required by the service-template, will be packaged 
into one archive. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (ARIA-213) Sporadic tests failures over locked database issue

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv closed ARIA-213.


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, statement, parameters, context=None):
> >   cursor.execute(statement, parameters)
> E   OperationalError: (sqlite3.OperationalError) database is locked [SQL: 
> u'SELECT task.id AS task_id, task.name AS task_name, task.implementation AS 
> task_implementation, task.max_attempts AS task_max_attempts, 
> task.retry_interval AS task_retry_interval, task.ignore_failure AS 
> task_ignore_failure, task.status AS task_status, task.due_at AS task_due_at, 
> task.started_at AS task_started_at, task.ended_at AS task_ended_at, 
> task.attempts_count AS 

[jira] [Created] (ARIA-248) When applying tracked changes fails no message is returned

[Maxim Orlov (JIRA)]

Maxim Orlov created ARIA-248:


 Summary: When applying tracked changes fails no message is returned
 Key: ARIA-248
 URL: https://issues.apache.org/jira/browse/ARIA-248
 Project: AriaTosca
  Issue Type: Bug
Reporter: Maxim Orlov


When the instumentation module tries to apply tracked changes, and it fails, it 
should print out the attributes it did manage to update. However, if one of 
those attributes is a _STUB (which is an object() type object), json.dumps 
fails to decode it. 
https://github.com/apache/incubator-ariatosca/blob/master/aria/storage/instrumentation.py#L259

This prevents from any msg to be returned.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006533#comment-16006533
 ] 

ASF subversion and git services commented on ARIA-213:
--

Commit 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589 in incubator-ariatosca's branch 
refs/heads/master from [~Mxmrlv]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=2ee06b8 ]

ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006536#comment-16006536
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/129


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, statement, parameters, context=None):
> >   cursor.execute(statement, parameters)
> E   OperationalError: (sqlite3.OperationalError) database is locked [SQL: 
> u'SELECT task.id AS task_id, task.name AS task_name, task.implementation AS 
> task_implementation, task.max_attempts AS task_max_attempts, 
> task.retry_interval AS task_retry_interval, task.ignore_failure AS 
> 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/129


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

incubator-ariatosca git commit: ARIA-213 Sporadic tests failures over locked database issue

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/master 6864d42e1 -> 2ee06b8a6


ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/2ee06b8a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/2ee06b8a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/2ee06b8a

Branch: refs/heads/master
Commit: 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589
Parents: 6864d42
Author: max-orlov 
Authored: Tue May 9 17:24:31 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 17:24:15 2017 +0300

--
 aria/logger.py  | 25 +
 aria/orchestrator/context/common.py | 13 +--
 .../workflows/core/events_handler.py|  4 +-
 aria/orchestrator/workflows/executor/process.py | 51 ++
 aria/storage/instrumentation.py | 97 +---
 .../orchestrator/workflows/executor/__init__.py | 21 -
 .../workflows/executor/test_executor.py | 36 ++--
 .../workflows/executor/test_process_executor.py | 13 ++-
 tests/storage/test_instrumentation.py   | 37 +++-
 9 files changed, 219 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 97d3878..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -114,17 +114,11 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
-
-return _SQLAlchemyHandler(session=session,
-  engine=engine,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):
@@ -168,10 +162,9 @@ def create_file_log_handler(
 
 class _SQLAlchemyHandler(logging.Handler):
 
-def __init__(self, session, engine, log_cls, execution_id, **kwargs):
+def __init__(self, model, log_cls, execution_id, **kwargs):
 logging.Handler.__init__(self, **kwargs)
-self._session = session
-self._engine = engine
+self._model = model
 self._cls = log_cls
 self._execution_id = execution_id
 
@@ -188,15 +181,7 @@ class _SQLAlchemyHandler(logging.Handler):
 # Not mandatory.
 traceback=getattr(record, 'traceback', None)
 )
-self._session.add(log)
-
-try:
-self._session.commit()
-except BaseException:
-self._session.rollback()
-raise
-finally:
-self._session.close()
+self._model.log.put(log)
 
 
 _default_file_formatter = logging.Formatter(

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/orchestrator/context/common.py
--
diff --git a/aria/orchestrator/context/common.py 
b/aria/orchestrator/context/common.py
index 64ef9a4..0854a27 100644
--- a/aria/orchestrator/context/common.py
+++ b/aria/orchestrator/context/common.py
@@ -79,13 +79,9 @@ class BaseContext(object):
 self.logger.addHandler(self._get_sqla_handler())
 
 def _get_sqla_handler(self):
-api_kwargs = {}
-if self._model._initiator:
-
api_kwargs.update(self._model._initiator(**self._model._initiator_kwargs))
-api_kwargs.update(**self._model._api_kwargs)
-return aria_logger.create_sqla_log_handler(log_cls=modeling.models.Log,
-   
execution_id=self._execution_id,
-   **api_kwargs)
+return aria_logger.create_sqla_log_handler(model=self._model,
+   log_cls=modeling.models.Log,
+   

[jira] [Created] (ARIA-247) Automatically inferring single-package plugins package prefix

[Ran Ziv (JIRA)]

Ran Ziv created ARIA-247:


 Summary: Automatically inferring single-package plugins package 
prefix
 Key: ARIA-247
 URL: https://issues.apache.org/jira/browse/ARIA-247
 Project: AriaTosca
  Issue Type: Story
Reporter: Ran Ziv
Priority: Trivial


When mapping an operation implementation in the service template to some 
plugin, one would usually map it like so:

{code}
implementation: my_plugin > my_package.my_module.my_function
{code}

We could improve on this and possibly help masking the "Python" aspect of this 
path if the {{PluginManager}} class, when loading a plugin, would automatically 
infer the {{my_package}} prefix when it comes to single-package plugins (a 
common case).

That would allow for the implementation to also look like this:

{code}
implementation: my_plugin > my_module.my_function
{code}

Then, plugin authors could also alias operations entry points in the package's 
{{__init__.py}} file, and thereby allowing the implementation line so simply be:

{code}
implementation: my_plugin > my_function
{code}

Which is much neater.

(Note that supporting both cases, i.e. both inferring and also trying the exact 
implementation string, may cause ambiguity in some scenarios)




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ARIA-247) Automatically infer single-package plugins package prefix

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIA-247:
-
Summary: Automatically infer single-package plugins package prefix  (was: 
Automatically inferring single-package plugins package prefix)

> Automatically infer single-package plugins package prefix
> -
>
> Key: ARIA-247
> URL: https://issues.apache.org/jira/browse/ARIA-247
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Priority: Trivial
>
> When mapping an operation implementation in the service template to some 
> plugin, one would usually map it like so:
> {code}
> implementation: my_plugin > my_package.my_module.my_function
> {code}
> We could improve on this and possibly help masking the "Python" aspect of 
> this path if the {{PluginManager}} class, when loading a plugin, would 
> automatically infer the {{my_package}} prefix when it comes to single-package 
> plugins (a common case).
> That would allow for the implementation to also look like this:
> {code}
> implementation: my_plugin > my_module.my_function
> {code}
> Then, plugin authors could also alias operations entry points in the 
> package's {{__init__.py}} file, and thereby allowing the implementation line 
> so simply be:
> {code}
> implementation: my_plugin > my_function
> {code}
> Which is much neater.
> (Note that supporting both cases, i.e. both inferring and also trying the 
> exact implementation string, may cause ambiguity in some scenarios)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ARIA-246) Built-in workflows not appear on workflows list

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIA-246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIA-246:
-
Description: 
The built-in workflows are not stored on the service model's {{workflow}} field.

This can make sense in some ways, but can also cause awkward behavior, such as 
having an empty list returned when listing workflows in the CLI for a service 
which has no custom workflows.

It might be better to partially or fully push the built-in workflows into the 
service model's {{workflows}} field for it to be more whole.
This could also simplify some code in {{WorkflowRunner}} as there'd be no need 
to separate the cases of custom and built-in workflows anymore (except for, 
possibly, when referring to the workflow implementation)

  was:
The built-in workflows are not stored on the service model's {{workflow}} field.

This can make sense in some ways, but can also cause awkward behavior, such as 
having an empty list returned when listing workflows in the CLI for a service 
which has no custom workflows.

It might be better to partially or fully push the built-in workflows into the 
service model's {{workflows}} field for it to be more whole.


> Built-in workflows not appear on workflows list
> ---
>
> Key: ARIA-246
> URL: https://issues.apache.org/jira/browse/ARIA-246
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Priority: Trivial
>
> The built-in workflows are not stored on the service model's {{workflow}} 
> field.
> This can make sense in some ways, but can also cause awkward behavior, such 
> as having an empty list returned when listing workflows in the CLI for a 
> service which has no custom workflows.
> It might be better to partially or fully push the built-in workflows into the 
> service model's {{workflows}} field for it to be more whole.
> This could also simplify some code in {{WorkflowRunner}} as there'd be no 
> need to separate the cases of custom and built-in workflows anymore (except 
> for, possibly, when referring to the workflow implementation)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ARIA-246) Built-in workflows not appear on workflows list

[Ran Ziv (JIRA)]

Ran Ziv created ARIA-246:


 Summary: Built-in workflows not appear on workflows list
 Key: ARIA-246
 URL: https://issues.apache.org/jira/browse/ARIA-246
 Project: AriaTosca
  Issue Type: Story
Reporter: Ran Ziv
Priority: Trivial


The built-in workflows are not stored on the service model's {{workflow}} field.

This can make sense in some ways, but can also cause awkward behavior, such as 
having an empty list returned when listing workflows in the CLI for a service 
which has no custom workflows.

It might be better to partially or fully push the built-in workflows into the 
service model's {{workflows}} field for it to be more whole.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006509#comment-16006509
 ] 

ASF subversion and git services commented on ARIA-213:
--

Commit 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589 in incubator-ariatosca's branch 
refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue from 
[~Mxmrlv]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=2ee06b8 ]

ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def 

incubator-ariatosca git commit: ARIA-213 Sporadic tests failures over locked database issue [Forced Update!]

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue 
2920661ac -> 2ee06b8a6 (forced update)


ARIA-213 Sporadic tests failures over locked database issue

Move from 2 different sessions - one for the log, and the other for general 
model operations,
to one single session, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/2ee06b8a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/2ee06b8a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/2ee06b8a

Branch: refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue
Commit: 2ee06b8a6abe79f429458c7dbc5f9e1c31aec589
Parents: 6864d42
Author: max-orlov 
Authored: Tue May 9 17:24:31 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 17:24:15 2017 +0300

--
 aria/logger.py  | 25 +
 aria/orchestrator/context/common.py | 13 +--
 .../workflows/core/events_handler.py|  4 +-
 aria/orchestrator/workflows/executor/process.py | 51 ++
 aria/storage/instrumentation.py | 97 +---
 .../orchestrator/workflows/executor/__init__.py | 21 -
 .../workflows/executor/test_executor.py | 36 ++--
 .../workflows/executor/test_process_executor.py | 13 ++-
 tests/storage/test_instrumentation.py   | 37 +++-
 9 files changed, 219 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 97d3878..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -114,17 +114,11 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
-
-return _SQLAlchemyHandler(session=session,
-  engine=engine,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):
@@ -168,10 +162,9 @@ def create_file_log_handler(
 
 class _SQLAlchemyHandler(logging.Handler):
 
-def __init__(self, session, engine, log_cls, execution_id, **kwargs):
+def __init__(self, model, log_cls, execution_id, **kwargs):
 logging.Handler.__init__(self, **kwargs)
-self._session = session
-self._engine = engine
+self._model = model
 self._cls = log_cls
 self._execution_id = execution_id
 
@@ -188,15 +181,7 @@ class _SQLAlchemyHandler(logging.Handler):
 # Not mandatory.
 traceback=getattr(record, 'traceback', None)
 )
-self._session.add(log)
-
-try:
-self._session.commit()
-except BaseException:
-self._session.rollback()
-raise
-finally:
-self._session.close()
+self._model.log.put(log)
 
 
 _default_file_formatter = logging.Formatter(

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ee06b8a/aria/orchestrator/context/common.py
--
diff --git a/aria/orchestrator/context/common.py 
b/aria/orchestrator/context/common.py
index 64ef9a4..0854a27 100644
--- a/aria/orchestrator/context/common.py
+++ b/aria/orchestrator/context/common.py
@@ -79,13 +79,9 @@ class BaseContext(object):
 self.logger.addHandler(self._get_sqla_handler())
 
 def _get_sqla_handler(self):
-api_kwargs = {}
-if self._model._initiator:
-
api_kwargs.update(self._model._initiator(**self._model._initiator_kwargs))
-api_kwargs.update(**self._model._api_kwargs)
-return aria_logger.create_sqla_log_handler(log_cls=modeling.models.Log,
-   
execution_id=self._execution_id,
-   **api_kwargs)
+return aria_logger.create_sqla_log_handler(model=self._model,
+  

[jira] [Created] (ARIA-245) Dry execution of operations

[Ran Ziv (JIRA)]

Ran Ziv created ARIA-245:


 Summary: Dry execution of operations
 Key: ARIA-245
 URL: https://issues.apache.org/jira/browse/ARIA-245
 Project: AriaTosca
  Issue Type: Story
Reporter: Ran Ziv
Priority: Trivial


It is currently possible to run a dry execution of a workflow using the 
{{DryExecutor}}. This execution will behave exactly like a real execution, 
including validating the execution inputs, building the workflow graph, and 
(dry-)executing the tasks in order.

However, the operations themselves don't actually get called, and thus the dry 
run isn't "complete". Plugins might want to have their own output for dry 
execution of operations.

Therefore, it might be a good idea to have a standard plugin policy property 
which a plugin could override to declare that it supports dry execution of 
operations - in which case, the dry executor would actually call the operation, 
providing a flag describing that it's meant to only dry-run.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006487#comment-16006487
 ] 

ASF subversion and git services commented on ARIA-213:
--

Commit 2920661ac63a2ceb937ae8caf75b610968bdeb11 in incubator-ariatosca's branch 
refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue from 
[~Mxmrlv]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=2920661 ]

ARIA-213 Sporadic tests failures over locked database issue

In this PR we moved from 2 different sessions - one for the log, and the other 
for general model operations,
to one single sessions, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  

incubator-ariatosca git commit: ARIA-213 Sporadic tests failures over locked database issue [Forced Update!]

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue 
0632e07c2 -> 2920661ac (forced update)


ARIA-213 Sporadic tests failures over locked database issue

In this PR we moved from 2 different sessions - one for the log, and the other 
for general model operations,
to one single sessions, while utilizing the keep tracking of changes mechanism 
for both logs and node/task states.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/2920661a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/2920661a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/2920661a

Branch: refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue
Commit: 2920661ac63a2ceb937ae8caf75b610968bdeb11
Parents: 6864d42
Author: max-orlov 
Authored: Tue May 9 17:24:31 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 17:14:52 2017 +0300

--
 aria/logger.py  | 25 +
 aria/orchestrator/context/common.py | 13 +--
 .../workflows/core/events_handler.py|  4 +-
 aria/orchestrator/workflows/executor/process.py | 51 ++
 aria/storage/instrumentation.py | 97 +---
 .../orchestrator/workflows/executor/__init__.py | 21 -
 .../workflows/executor/test_executor.py | 36 ++--
 .../workflows/executor/test_process_executor.py | 13 ++-
 tests/storage/test_instrumentation.py   | 37 +++-
 9 files changed, 219 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2920661a/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 97d3878..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -114,17 +114,11 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
-
-return _SQLAlchemyHandler(session=session,
-  engine=engine,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):
@@ -168,10 +162,9 @@ def create_file_log_handler(
 
 class _SQLAlchemyHandler(logging.Handler):
 
-def __init__(self, session, engine, log_cls, execution_id, **kwargs):
+def __init__(self, model, log_cls, execution_id, **kwargs):
 logging.Handler.__init__(self, **kwargs)
-self._session = session
-self._engine = engine
+self._model = model
 self._cls = log_cls
 self._execution_id = execution_id
 
@@ -188,15 +181,7 @@ class _SQLAlchemyHandler(logging.Handler):
 # Not mandatory.
 traceback=getattr(record, 'traceback', None)
 )
-self._session.add(log)
-
-try:
-self._session.commit()
-except BaseException:
-self._session.rollback()
-raise
-finally:
-self._session.close()
+self._model.log.put(log)
 
 
 _default_file_formatter = logging.Formatter(

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2920661a/aria/orchestrator/context/common.py
--
diff --git a/aria/orchestrator/context/common.py 
b/aria/orchestrator/context/common.py
index 64ef9a4..0854a27 100644
--- a/aria/orchestrator/context/common.py
+++ b/aria/orchestrator/context/common.py
@@ -79,13 +79,9 @@ class BaseContext(object):
 self.logger.addHandler(self._get_sqla_handler())
 
 def _get_sqla_handler(self):
-api_kwargs = {}
-if self._model._initiator:
-
api_kwargs.update(self._model._initiator(**self._model._initiator_kwargs))
-api_kwargs.update(**self._model._api_kwargs)
-return aria_logger.create_sqla_log_handler(log_cls=modeling.models.Log,
-   
execution_id=self._execution_id,
-   **api_kwargs)
+return aria_logger.create_sqla_log_handler(model=self._model,
+  

incubator-ariatosca git commit: code review 1 [Forced Update!]

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue 
d3a84384d -> 0632e07c2 (forced update)


code review 1


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/0632e07c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/0632e07c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/0632e07c

Branch: refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue
Commit: 0632e07c2fe94ed73a6b0aa6d0c17493a1355c9f
Parents: a37af2a
Author: max-orlov 
Authored: Thu May 11 16:45:35 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 16:55:59 2017 +0300

--
 aria/logger.py  |  7 +-
 aria/orchestrator/workflows/executor/process.py |  7 +-
 aria/storage/instrumentation.py | 68 +++-
 .../workflows/executor/test_executor.py |  2 +-
 tests/storage/test_instrumentation.py   |  9 +--
 5 files changed, 45 insertions(+), 48 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0632e07c/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 9214bd9..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -118,12 +118,7 @@ def create_sqla_log_handler(model, log_cls, execution_id, 
level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=model.log._engine, checkfirst=True)
-
-return _SQLAlchemyHandler(model=model,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0632e07c/aria/orchestrator/workflows/executor/process.py
--
diff --git a/aria/orchestrator/workflows/executor/process.py 
b/aria/orchestrator/workflows/executor/process.py
index e8cf019..824c4e1 100644
--- a/aria/orchestrator/workflows/executor/process.py
+++ b/aria/orchestrator/workflows/executor/process.py
@@ -340,7 +340,7 @@ def _patch_ctx(ctx, messenger, instrument):
 
 def patched_rollback():
 # Rollback is performed on parent process when commit fails
-pass
+instrument.expunge_session()
 
 # when autoflush is set to true (the default), refreshing an object will 
trigger
 # an auto flush by sqlalchemy, this autoflush will attempt to commit 
changes made so
@@ -378,7 +378,7 @@ def _main():
 messenger.failed(exception=e, tracked_changes=None, new_instances=None)
 return
 
-with instrumentation.track_changes(ctx) as instrument:
+with instrumentation.track_changes(ctx.model) as instrument:
 try:
 messenger.started()
 _patch_ctx(ctx=ctx, messenger=messenger, instrument=instrument)
@@ -393,7 +393,8 @@ def _main():
 messenger.failed(exception=e,
  tracked_changes=instrument.tracked_changes,
  new_instances=instrument.new_instances)
-
+finally:
+instrument.expunge_session()
 
 if __name__ == '__main__':
 _main()

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0632e07c/aria/storage/instrumentation.py
--
diff --git a/aria/storage/instrumentation.py b/aria/storage/instrumentation.py
index 6af9473..390f933 100644
--- a/aria/storage/instrumentation.py
+++ b/aria/storage/instrumentation.py
@@ -14,6 +14,8 @@
 # limitations under the License.
 
 import copy
+import json
+import os
 
 import sqlalchemy.event
 
@@ -36,7 +38,7 @@ _INSTRUMENTED = {
 _NEW_INSTANCE = 'NEW_INSTANCE'
 
 
-def track_changes(ctx=None, instrumented=None):
+def track_changes(model=None, instrumented=None):
 """Track changes in the specified model columns
 
 This call will register event listeners using sqlalchemy's event 
mechanism. The listeners
@@ -56,40 +58,42 @@ def track_changes(ctx=None, instrumented=None):
 will then call ``apply_tracked_changes()`` that resides in this module as 
well.
 At that point, the changes will actually be written back to the database.
 
+:param model: the model storage. it should hold a mapi for each model. the 
session of each mapi
+is needed to setup events
 

incubator-ariatosca git commit: code review 1 [Forced Update!]

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue 
9d0f37ec5 -> d3a84384d (forced update)


code review 1


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/d3a84384
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/d3a84384
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/d3a84384

Branch: refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue
Commit: d3a84384dfde648da6f9f61bc781e07dccc5dda7
Parents: a37af2a
Author: max-orlov 
Authored: Thu May 11 16:45:35 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 16:49:32 2017 +0300

--
 aria/logger.py  |  7 +-
 aria/orchestrator/workflows/executor/process.py |  7 +-
 aria/storage/instrumentation.py | 67 +++-
 .../workflows/executor/test_executor.py |  2 +-
 tests/storage/test_instrumentation.py   |  8 +--
 5 files changed, 45 insertions(+), 46 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d3a84384/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 9214bd9..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -118,12 +118,7 @@ def create_sqla_log_handler(model, log_cls, execution_id, 
level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=model.log._engine, checkfirst=True)
-
-return _SQLAlchemyHandler(model=model,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d3a84384/aria/orchestrator/workflows/executor/process.py
--
diff --git a/aria/orchestrator/workflows/executor/process.py 
b/aria/orchestrator/workflows/executor/process.py
index e8cf019..824c4e1 100644
--- a/aria/orchestrator/workflows/executor/process.py
+++ b/aria/orchestrator/workflows/executor/process.py
@@ -340,7 +340,7 @@ def _patch_ctx(ctx, messenger, instrument):
 
 def patched_rollback():
 # Rollback is performed on parent process when commit fails
-pass
+instrument.expunge_session()
 
 # when autoflush is set to true (the default), refreshing an object will 
trigger
 # an auto flush by sqlalchemy, this autoflush will attempt to commit 
changes made so
@@ -378,7 +378,7 @@ def _main():
 messenger.failed(exception=e, tracked_changes=None, new_instances=None)
 return
 
-with instrumentation.track_changes(ctx) as instrument:
+with instrumentation.track_changes(ctx.model) as instrument:
 try:
 messenger.started()
 _patch_ctx(ctx=ctx, messenger=messenger, instrument=instrument)
@@ -393,7 +393,8 @@ def _main():
 messenger.failed(exception=e,
  tracked_changes=instrument.tracked_changes,
  new_instances=instrument.new_instances)
-
+finally:
+instrument.expunge_session()
 
 if __name__ == '__main__':
 _main()

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d3a84384/aria/storage/instrumentation.py
--
diff --git a/aria/storage/instrumentation.py b/aria/storage/instrumentation.py
index 6af9473..315d6b8 100644
--- a/aria/storage/instrumentation.py
+++ b/aria/storage/instrumentation.py
@@ -14,6 +14,8 @@
 # limitations under the License.
 
 import copy
+import json
+import os
 
 import sqlalchemy.event
 
@@ -36,7 +38,7 @@ _INSTRUMENTED = {
 _NEW_INSTANCE = 'NEW_INSTANCE'
 
 
-def track_changes(ctx=None, instrumented=None):
+def track_changes(model=None, instrumented=None):
 """Track changes in the specified model columns
 
 This call will register event listeners using sqlalchemy's event 
mechanism. The listeners
@@ -55,41 +57,43 @@ def track_changes(ctx=None, instrumented=None):
 the instrumentation context returned from this call, to the parent 
process. The parent process
 will then call ``apply_tracked_changes()`` that resides in this module as 
well.
 At that point, the changes will actually be written back to the database.
-
+
+:param model: the model 

incubator-ariatosca git commit: code review 1

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue 
a37af2a8f -> 9d0f37ec5


code review 1


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/9d0f37ec
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/9d0f37ec
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/9d0f37ec

Branch: refs/heads/ARIA-213-Sporadic-tests-failures-over-locked-database-issue
Commit: 9d0f37ec569585206eee8a99756b02688e6f6527
Parents: a37af2a
Author: max-orlov 
Authored: Thu May 11 16:45:35 2017 +0300
Committer: max-orlov 
Committed: Thu May 11 16:45:35 2017 +0300

--
 aria/logger.py  |  7 +--
 aria/orchestrator/workflows/executor/process.py |  7 ++-
 aria/storage/instrumentation.py | 59 +++-
 .../workflows/executor/test_executor.py |  2 +-
 tests/storage/test_instrumentation.py   |  8 +--
 5 files changed, 41 insertions(+), 42 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/9d0f37ec/aria/logger.py
--
diff --git a/aria/logger.py b/aria/logger.py
index 9214bd9..bd7ed4e 100644
--- a/aria/logger.py
+++ b/aria/logger.py
@@ -118,12 +118,7 @@ def create_sqla_log_handler(model, log_cls, execution_id, 
level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need to 
reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=model.log._engine, checkfirst=True)
-
-return _SQLAlchemyHandler(model=model,
-  log_cls=log_cls,
-  execution_id=execution_id,
-  level=level)
+return _SQLAlchemyHandler(model=model, log_cls=log_cls, 
execution_id=execution_id, level=level)
 
 
 class _DefaultConsoleFormat(logging.Formatter):

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/9d0f37ec/aria/orchestrator/workflows/executor/process.py
--
diff --git a/aria/orchestrator/workflows/executor/process.py 
b/aria/orchestrator/workflows/executor/process.py
index e8cf019..824c4e1 100644
--- a/aria/orchestrator/workflows/executor/process.py
+++ b/aria/orchestrator/workflows/executor/process.py
@@ -340,7 +340,7 @@ def _patch_ctx(ctx, messenger, instrument):
 
 def patched_rollback():
 # Rollback is performed on parent process when commit fails
-pass
+instrument.expunge_session()
 
 # when autoflush is set to true (the default), refreshing an object will 
trigger
 # an auto flush by sqlalchemy, this autoflush will attempt to commit 
changes made so
@@ -378,7 +378,7 @@ def _main():
 messenger.failed(exception=e, tracked_changes=None, new_instances=None)
 return
 
-with instrumentation.track_changes(ctx) as instrument:
+with instrumentation.track_changes(ctx.model) as instrument:
 try:
 messenger.started()
 _patch_ctx(ctx=ctx, messenger=messenger, instrument=instrument)
@@ -393,7 +393,8 @@ def _main():
 messenger.failed(exception=e,
  tracked_changes=instrument.tracked_changes,
  new_instances=instrument.new_instances)
-
+finally:
+instrument.expunge_session()
 
 if __name__ == '__main__':
 _main()

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/9d0f37ec/aria/storage/instrumentation.py
--
diff --git a/aria/storage/instrumentation.py b/aria/storage/instrumentation.py
index 6af9473..23f6fd0 100644
--- a/aria/storage/instrumentation.py
+++ b/aria/storage/instrumentation.py
@@ -14,6 +14,8 @@
 # limitations under the License.
 
 import copy
+import json
+import os
 
 import sqlalchemy.event
 
@@ -36,7 +38,7 @@ _INSTRUMENTED = {
 _NEW_INSTANCE = 'NEW_INSTANCE'
 
 
-def track_changes(ctx=None, instrumented=None):
+def track_changes(model=None, instrumented=None):
 """Track changes in the specified model columns
 
 This call will register event listeners using sqlalchemy's event 
mechanism. The listeners
@@ -55,41 +57,43 @@ def track_changes(ctx=None, instrumented=None):
 the instrumentation context returned from this call, to the parent 
process. The parent process
 will then call ``apply_tracked_changes()`` that resides in this module as 
well.
 At that point, the changes will actually be written back to the database.
-
+
+:param model: the model storage. it should 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115981300
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
+instance = mapi.model_cls(**new_instance_kwargs)
+mapi.put(instance)
+successfully_updated_changes[mapi_name][instance.id] = 
new_instance_kwargs
 except BaseException:
 for key, value in successfully_updated_changes.items():
 if not value:
 del successfully_updated_changes[key]
-model.logger.error(
-'Registering all the changes to the storage has failed. {0}'
-'The successful updates were: {0} '
-'{1}'.format(os.linesep, 
json.dumps(successfully_updated_changes, indent=4)))
+# TODO: if the successful has _STUB, the logging fails because it 
can't serialize the object
--- End diff --

uncomment


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006410#comment-16006410
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115985412
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
--- End diff --

`instances_to_expunge`? document


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115978355
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
+for new_instance in self._new_modeled_instances:
+
self._get_session_from_ctx(new_instance.__tablename__).expunge(new_instance)
+
+def _get_session_from_ctx(self, tablename):
+mapi = getattr(self._ctx.model, tablename, None)
+if mapi:
+return mapi._session
+raise StorageError("Could not retrieve session for 
{0}".format(tablename))
 
 def _track_changes(self, instrumented):
-instrumented_classes = {}
-for instrumented_attribute, attribute_type in instrumented.items():
+instrumented_attribute_classes = {}
+for instrumented_attribute, attribute_type in 
instrumented.get('modified', {}).items():
 self._register_set_attribute_listener(
 instrumented_attribute=instrumented_attribute,
 attribute_type=attribute_type)
 instrumented_class = instrumented_attribute.parent.entity
-instrumented_class_attributes = 
instrumented_classes.setdefault(instrumented_class, {})
+instrumented_class_attributes = 
instrumented_attribute_classes.setdefault(
+instrumented_class, {})
 instrumented_class_attributes[instrumented_attribute.key] = 
attribute_type
-for instrumented_class, instrumented_attributes in 
instrumented_classes.items():
-self._register_instance_listeners(
+for instrumented_class, instrumented_attributes in 
instrumented_attribute_classes.items():
+self._register_instance_attribute_listeners(
 instrumented_class=instrumented_class,
 instrumented_attributes=instrumented_attributes)
 
+# instrument creation of new instances
+for instrumented_class in instrumented.get('new', {}):
+self._register_instance_listener(instrumented_class)
+
+def _register_instance_listener(self, instrumented_class):
+if self._ctx is None:
+if instrumented_class:
--- End diff --

remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006400#comment-16006400
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115973634
  
--- Diff: tests/orchestrator/workflows/executor/test_executor.py ---
@@ -41,19 +42,20 @@ def _get_implementation(func):
 return '{module}.{func.__name__}'.format(module=__name__, func=func)
 
 
-def test_execute(executor):
+def execute_and_assert(executor, storage=None):
 expected_value = 'value'
-successful_task = MockTask(_get_implementation(mock_successful_task))
-failing_task = MockTask(_get_implementation(mock_failing_task))
+successful_task = MockTask(_get_implementation(mock_successful_task), 
storage=storage)
+failing_task = MockTask(_get_implementation(mock_failing_task), 
storage=storage)
 task_with_inputs = MockTask(_get_implementation(mock_task_with_input),
-inputs={'input': 
models.Parameter.wrap('input', 'value')})
+inputs={'input': 
models.Parameter.wrap('input', 'value')},
+storage=storage)
 
 for task in [successful_task, failing_task, task_with_inputs]:
 executor.execute(task)
 
 @retrying.retry(stop_max_delay=1, wait_fixed=100)
 def assertion():
-assert successful_task.states == ['start', 'success']
+# assert successful_task.states == ['start', 'success']
--- End diff --

remove


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006401#comment-16006401
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115981331
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
+instance = mapi.model_cls(**new_instance_kwargs)
+mapi.put(instance)
+successfully_updated_changes[mapi_name][instance.id] = 
new_instance_kwargs
 except BaseException:
 for key, value in successfully_updated_changes.items():
 if not value:
 del successfully_updated_changes[key]
-model.logger.error(
-'Registering all the changes to the storage has failed. {0}'
-'The successful updates were: {0} '
-'{1}'.format(os.linesep, 
json.dumps(successfully_updated_changes, indent=4)))
+# TODO: if the successful has _STUB, the logging fails because it 
can't serialize the object
--- End diff --

open jira


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115981331
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
+instance = mapi.model_cls(**new_instance_kwargs)
+mapi.put(instance)
+successfully_updated_changes[mapi_name][instance.id] = 
new_instance_kwargs
 except BaseException:
 for key, value in successfully_updated_changes.items():
 if not value:
 del successfully_updated_changes[key]
-model.logger.error(
-'Registering all the changes to the storage has failed. {0}'
-'The successful updates were: {0} '
-'{1}'.format(os.linesep, 
json.dumps(successfully_updated_changes, indent=4)))
+# TODO: if the successful has _STUB, the logging fails because it 
can't serialize the object
--- End diff --

open jira


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006407#comment-16006407
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115982730
  
--- Diff: aria/storage/instrumentation.py ---
@@ -26,11 +24,19 @@
 _VERSION_ID_COL = 'version'
 _STUB = object()
 _INSTRUMENTED = {
-_models.Node.runtime_properties: dict
+'modified': {
+_models.Node.runtime_properties: dict,
+_models.Node.state: str,
+_models.Task.status: str,
+},
+'new': (_models.Log, )
+
 }
 
+_NEW_INSTANCE = 'NEW_INSTANCE'
+
 
-def track_changes(instrumented=None):
+def track_changes(ctx=None, instrumented=None):
--- End diff --

consider model


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115985223
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
--- End diff --

should also call expunge from rollback; also, what happens if the 
transaction is never completed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115983463
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
--- End diff --

remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115976779
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
--- End diff --

remove "-1"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006402#comment-16006402
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115985223
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
--- End diff --

should also call expunge from rollback; also, what happens if the 
transaction is never completed?


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006403#comment-16006403
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115973547
  
--- Diff: tests/storage/test_instrumentation.py ---
@@ -35,6 +36,7 @@
 instruments_holder = []
 
 
+# TODO: add testing for new instances
--- End diff --

remove


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, statement, parameters, context=None):
> >   cursor.execute(statement, parameters)
> E   OperationalError: (sqlite3.OperationalError) database is locked [SQL: 
> u'SELECT task.id AS 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006409#comment-16006409
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115976102
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
--- End diff --

document ctx


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, statement, parameters, 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115978241
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
+for new_instance in self._new_modeled_instances:
+
self._get_session_from_ctx(new_instance.__tablename__).expunge(new_instance)
+
+def _get_session_from_ctx(self, tablename):
+mapi = getattr(self._ctx.model, tablename, None)
+if mapi:
+return mapi._session
+raise StorageError("Could not retrieve session for 
{0}".format(tablename))
 
 def _track_changes(self, instrumented):
-instrumented_classes = {}
-for instrumented_attribute, attribute_type in instrumented.items():
+instrumented_attribute_classes = {}
+for instrumented_attribute, attribute_type in 
instrumented.get('modified', {}).items():
 self._register_set_attribute_listener(
 instrumented_attribute=instrumented_attribute,
 attribute_type=attribute_type)
 instrumented_class = instrumented_attribute.parent.entity
-instrumented_class_attributes = 
instrumented_classes.setdefault(instrumented_class, {})
+instrumented_class_attributes = 
instrumented_attribute_classes.setdefault(
+instrumented_class, {})
 instrumented_class_attributes[instrumented_attribute.key] = 
attribute_type
-for instrumented_class, instrumented_attributes in 
instrumented_classes.items():
-self._register_instance_listeners(
+for instrumented_class, instrumented_attributes in 
instrumented_attribute_classes.items():
+self._register_instance_attribute_listeners(
 instrumented_class=instrumented_class,
 instrumented_attributes=instrumented_attributes)
 
+# instrument creation of new instances
+for instrumented_class in instrumented.get('new', {}):
+self._register_instance_listener(instrumented_class)
--- End diff --

add documentation explaining what is what


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006406#comment-16006406
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115980345
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
--- End diff --

commen
why sorted


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006408#comment-16006408
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115978355
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
+for new_instance in self._new_modeled_instances:
+
self._get_session_from_ctx(new_instance.__tablename__).expunge(new_instance)
+
+def _get_session_from_ctx(self, tablename):
+mapi = getattr(self._ctx.model, tablename, None)
+if mapi:
+return mapi._session
+raise StorageError("Could not retrieve session for 
{0}".format(tablename))
 
 def _track_changes(self, instrumented):
-instrumented_classes = {}
-for instrumented_attribute, attribute_type in instrumented.items():
+instrumented_attribute_classes = {}
+for instrumented_attribute, attribute_type in 
instrumented.get('modified', {}).items():
 self._register_set_attribute_listener(
 instrumented_attribute=instrumented_attribute,
 attribute_type=attribute_type)
 instrumented_class = instrumented_attribute.parent.entity
-instrumented_class_attributes = 
instrumented_classes.setdefault(instrumented_class, {})
+instrumented_class_attributes = 
instrumented_attribute_classes.setdefault(
+instrumented_class, {})
 instrumented_class_attributes[instrumented_attribute.key] = 
attribute_type
-for instrumented_class, instrumented_attributes in 
instrumented_classes.items():
-self._register_instance_listeners(
+for instrumented_class, instrumented_attributes in 
instrumented_attribute_classes.items():
+self._register_instance_attribute_listeners(
 instrumented_class=instrumented_class,
 instrumented_attributes=instrumented_attributes)
 
+# instrument creation of new instances
+for instrumented_class in instrumented.get('new', {}):
+self._register_instance_listener(instrumented_class)
+
+def _register_instance_listener(self, instrumented_class):
+if self._ctx is None:
+if instrumented_class:
--- End diff --

remove


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006412#comment-16006412
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115976779
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
--- End diff --

remove "-1"


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115973547
  
--- Diff: tests/storage/test_instrumentation.py ---
@@ -35,6 +36,7 @@
 instruments_holder = []
 
 
+# TODO: add testing for new instances
--- End diff --

remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006399#comment-16006399
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115972098
  
--- Diff: aria/logger.py ---
@@ -114,14 +114,13 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, 
level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need 
to reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
+log_cls.__table__.create(bind=model.log._engine, checkfirst=True)
--- End diff --

remove?


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006404#comment-16006404
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115981300
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
+instance = mapi.model_cls(**new_instance_kwargs)
+mapi.put(instance)
+successfully_updated_changes[mapi_name][instance.id] = 
new_instance_kwargs
 except BaseException:
 for key, value in successfully_updated_changes.items():
 if not value:
 del successfully_updated_changes[key]
-model.logger.error(
-'Registering all the changes to the storage has failed. {0}'
-'The successful updates were: {0} '
-'{1}'.format(os.linesep, 
json.dumps(successfully_updated_changes, indent=4)))
+# TODO: if the successful has _STUB, the logging fails because it 
can't serialize the object
--- End diff --

uncomment


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> 

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115982730
  
--- Diff: aria/storage/instrumentation.py ---
@@ -26,11 +24,19 @@
 _VERSION_ID_COL = 'version'
 _STUB = object()
 _INSTRUMENTED = {
-_models.Node.runtime_properties: dict
+'modified': {
+_models.Node.runtime_properties: dict,
+_models.Node.state: str,
+_models.Task.status: str,
+},
+'new': (_models.Log, )
+
 }
 
+_NEW_INSTANCE = 'NEW_INSTANCE'
+
 
-def track_changes(instrumented=None):
+def track_changes(ctx=None, instrumented=None):
--- End diff --

consider model


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #129: ARIA-213 Sporadic tests failures over...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115972098
  
--- Diff: aria/logger.py ---
@@ -114,14 +114,13 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, 
level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need 
to reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
+log_cls.__table__.create(bind=model.log._engine, checkfirst=True)
--- End diff --

remove?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---