[jira] [Commented] (ARIA-171) Service_template.services type inconsistency

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010361#comment-16010361
 ] 

ASF GitHub Bot commented on ARIA-171:
-

GitHub user AviaE opened a pull request:

https://github.com/apache/incubator-ariatosca/pull/132

ARIA-171 service_template.services type inconsistency

As opposed to service_template.node_templates and service.nodes
which have a dict interface, service_template.services
had a list interface. That seemed as inconsistent, so the former
interface was changed to a dict interface as well.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/incubator-ariatosca 
ARIA-171-service-template-services-type-inconsistency

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-ariatosca/pull/132.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #132


commit 4425c10306517cffce930b3f81623f350a116fae
Author: Avia Efrat 
Date:   2017-05-15T11:26:34Z

ARIA-171 service_template.services type inconsistency

As opposed to service_template.node_templates and service.nodes
which have a dict interface, service_template.services
had a list interface. That seemed as inconsistent, so the former
interface was changed to a dict interface as well.




> Service_template.services type inconsistency
> 
>
> Key: ARIA-171
> URL: https://issues.apache.org/jira/browse/ARIA-171
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> {{service_template.services}} is an object of type {{InstrumentedList}}, 
> which is inconsistent with similar forms of model access (e.g. 
> {{service.nodes}}).
> The object's type should instead be a {{MappedCollection}} from the service 
> name to the service model object.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-139) TOSCA attributes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006759#comment-16006759
 ] 

ASF GitHub Bot commented on ARIA-139:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/118


> TOSCA attributes
> 
>
> Key: ARIA-139
> URL: https://issues.apache.org/jira/browse/ARIA-139
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> This feature needs to work in the parser, in modeling, and for the intrinsic 
> function {{get_attribute}}.
> We also need to find a way to relate existing {{runtime_properties}} to this 
> feature.
> We should convert our code that uses {{state}} field in nodes to use the 
> {{state}} attribute instead.
> We need to set the {{tosca_id}} and {{tosca_name}} attributes.
> We should use the {{public_address}} and {{private_address}} attributes 
> instead of the {{host_address}} property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006536#comment-16006536
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/129


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, statement, parameters, context=None):
> >   cursor.execute(statement, parameters)
> E   OperationalError: (sqlite3.OperationalError) database is locked [SQL: 
> u'SELECT task.id AS task_id, task.name AS task_name, task.implementation AS 
> task_implementation, task.max_attempts AS task_max_attempts, 
> task.retry_interval AS task_retry_interval, task.ignore_failure AS 
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006410#comment-16006410
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115985412
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
--- End diff --

`instances_to_expunge`? document


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006400#comment-16006400
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115973634
  
--- Diff: tests/orchestrator/workflows/executor/test_executor.py ---
@@ -41,19 +42,20 @@ def _get_implementation(func):
 return '{module}.{func.__name__}'.format(module=__name__, func=func)
 
 
-def test_execute(executor):
+def execute_and_assert(executor, storage=None):
 expected_value = 'value'
-successful_task = MockTask(_get_implementation(mock_successful_task))
-failing_task = MockTask(_get_implementation(mock_failing_task))
+successful_task = MockTask(_get_implementation(mock_successful_task), 
storage=storage)
+failing_task = MockTask(_get_implementation(mock_failing_task), 
storage=storage)
 task_with_inputs = MockTask(_get_implementation(mock_task_with_input),
-inputs={'input': 
models.Parameter.wrap('input', 'value')})
+inputs={'input': 
models.Parameter.wrap('input', 'value')},
+storage=storage)
 
 for task in [successful_task, failing_task, task_with_inputs]:
 executor.execute(task)
 
 @retrying.retry(stop_max_delay=1, wait_fixed=100)
 def assertion():
-assert successful_task.states == ['start', 'success']
+# assert successful_task.states == ['start', 'success']
--- End diff --

remove


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006401#comment-16006401
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115981331
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
+instance = mapi.model_cls(**new_instance_kwargs)
+mapi.put(instance)
+successfully_updated_changes[mapi_name][instance.id] = 
new_instance_kwargs
 except BaseException:
 for key, value in successfully_updated_changes.items():
 if not value:
 del successfully_updated_changes[key]
-model.logger.error(
-'Registering all the changes to the storage has failed. {0}'
-'The successful updates were: {0} '
-'{1}'.format(os.linesep, 
json.dumps(successfully_updated_changes, indent=4)))
+# TODO: if the successful has _STUB, the logging fails because it 
can't serialize the object
--- End diff --

open jira


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006407#comment-16006407
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115982730
  
--- Diff: aria/storage/instrumentation.py ---
@@ -26,11 +24,19 @@
 _VERSION_ID_COL = 'version'
 _STUB = object()
 _INSTRUMENTED = {
-_models.Node.runtime_properties: dict
+'modified': {
+_models.Node.runtime_properties: dict,
+_models.Node.state: str,
+_models.Task.status: str,
+},
+'new': (_models.Log, )
+
 }
 
+_NEW_INSTANCE = 'NEW_INSTANCE'
+
 
-def track_changes(instrumented=None):
+def track_changes(ctx=None, instrumented=None):
--- End diff --

consider model


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006402#comment-16006402
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115985223
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
--- End diff --

should also call expunge from rollback; also, what happens if the 
transaction is never completed?


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006403#comment-16006403
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115973547
  
--- Diff: tests/storage/test_instrumentation.py ---
@@ -35,6 +36,7 @@
 instruments_holder = []
 
 
+# TODO: add testing for new instances
--- End diff --

remove


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, statement, parameters, context=None):
> >   cursor.execute(statement, parameters)
> E   OperationalError: (sqlite3.OperationalError) database is locked [SQL: 
> u'SELECT task.id AS 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006409#comment-16006409
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115976102
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
--- End diff --

document ctx


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t...hip_fk 
> AS task_relationship_fk, task.execution_fk AS task_execution_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f2768227590>
> def do_execute(self, cursor, statement, parameters, 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006406#comment-16006406
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115980345
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
--- End diff --

commen
why sorted


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006408#comment-16006408
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115978355
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
+for new_instance in self._new_modeled_instances:
+
self._get_session_from_ctx(new_instance.__tablename__).expunge(new_instance)
+
+def _get_session_from_ctx(self, tablename):
+mapi = getattr(self._ctx.model, tablename, None)
+if mapi:
+return mapi._session
+raise StorageError("Could not retrieve session for 
{0}".format(tablename))
 
 def _track_changes(self, instrumented):
-instrumented_classes = {}
-for instrumented_attribute, attribute_type in instrumented.items():
+instrumented_attribute_classes = {}
+for instrumented_attribute, attribute_type in 
instrumented.get('modified', {}).items():
 self._register_set_attribute_listener(
 instrumented_attribute=instrumented_attribute,
 attribute_type=attribute_type)
 instrumented_class = instrumented_attribute.parent.entity
-instrumented_class_attributes = 
instrumented_classes.setdefault(instrumented_class, {})
+instrumented_class_attributes = 
instrumented_attribute_classes.setdefault(
+instrumented_class, {})
 instrumented_class_attributes[instrumented_attribute.key] = 
attribute_type
-for instrumented_class, instrumented_attributes in 
instrumented_classes.items():
-self._register_instance_listeners(
+for instrumented_class, instrumented_attributes in 
instrumented_attribute_classes.items():
+self._register_instance_attribute_listeners(
 instrumented_class=instrumented_class,
 instrumented_attributes=instrumented_attributes)
 
+# instrument creation of new instances
+for instrumented_class in instrumented.get('new', {}):
+self._register_instance_listener(instrumented_class)
+
+def _register_instance_listener(self, instrumented_class):
+if self._ctx is None:
+if instrumented_class:
--- End diff --

remove


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006412#comment-16006412
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115976779
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
--- End diff --

remove "-1"


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006399#comment-16006399
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115972098
  
--- Diff: aria/logger.py ---
@@ -114,14 +114,13 @@ def create_console_log_handler(level=logging.DEBUG, 
formatter=None):
 return console
 
 
-def create_sqla_log_handler(session, engine, log_cls, execution_id, 
level=logging.DEBUG):
+def create_sqla_log_handler(model, log_cls, execution_id, 
level=logging.DEBUG):
 
 # This is needed since the engine and session are entirely new we need 
to reflect the db
 # schema of the logging model into the engine and session.
-log_cls.__table__.create(bind=engine, checkfirst=True)
+log_cls.__table__.create(bind=model.log._engine, checkfirst=True)
--- End diff --

remove?


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f27683238d0>
> cursor = 
> statement = 'SELECT task.id AS task_id, 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006404#comment-16006404
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115981300
  
--- Diff: aria/storage/instrumentation.py ---
@@ -177,22 +232,30 @@ def apply_tracked_changes(tracked_changes, model):
 instance = None
 for attribute_name, value in tracked_attributes.items():
 if value.initial != value.current:
-if not instance:
-instance = mapi.get(instance_id)
+instance = instance or mapi.get(instance_id)
 setattr(instance, attribute_name, value.current)
 if instance:
 _validate_version_id(instance, mapi)
 mapi.update(instance)
 successfully_updated_changes[mapi_name][instance_id] = 
[
 v.dict for v in tracked_attributes.values()]
+
+for mapi_name, new_instance in new_instances.items():
+successfully_updated_changes[mapi_name] = dict()
+mapi = getattr(model, mapi_name)
+for new_instance_kwargs in sorted(new_instance.values()):
+instance = mapi.model_cls(**new_instance_kwargs)
+mapi.put(instance)
+successfully_updated_changes[mapi_name][instance.id] = 
new_instance_kwargs
 except BaseException:
 for key, value in successfully_updated_changes.items():
 if not value:
 del successfully_updated_changes[key]
-model.logger.error(
-'Registering all the changes to the storage has failed. {0}'
-'The successful updates were: {0} '
-'{1}'.format(os.linesep, 
json.dumps(successfully_updated_changes, indent=4)))
+# TODO: if the successful has _STUB, the logging fails because it 
can't serialize the object
--- End diff --

uncomment


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006411#comment-16006411
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115978241
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
+
+@property
+def _new_instance_id(self):
+self._new_instance_index += 1
+return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, 
index=self._new_instance_index - 1)
+
+def expunge_session(self):
+for new_instance in self._new_modeled_instances:
+
self._get_session_from_ctx(new_instance.__tablename__).expunge(new_instance)
+
+def _get_session_from_ctx(self, tablename):
+mapi = getattr(self._ctx.model, tablename, None)
+if mapi:
+return mapi._session
+raise StorageError("Could not retrieve session for 
{0}".format(tablename))
 
 def _track_changes(self, instrumented):
-instrumented_classes = {}
-for instrumented_attribute, attribute_type in instrumented.items():
+instrumented_attribute_classes = {}
+for instrumented_attribute, attribute_type in 
instrumented.get('modified', {}).items():
 self._register_set_attribute_listener(
 instrumented_attribute=instrumented_attribute,
 attribute_type=attribute_type)
 instrumented_class = instrumented_attribute.parent.entity
-instrumented_class_attributes = 
instrumented_classes.setdefault(instrumented_class, {})
+instrumented_class_attributes = 
instrumented_attribute_classes.setdefault(
+instrumented_class, {})
 instrumented_class_attributes[instrumented_attribute.key] = 
attribute_type
-for instrumented_class, instrumented_attributes in 
instrumented_classes.items():
-self._register_instance_listeners(
+for instrumented_class, instrumented_attributes in 
instrumented_attribute_classes.items():
+self._register_instance_attribute_listeners(
 instrumented_class=instrumented_class,
 instrumented_attributes=instrumented_attributes)
 
+# instrument creation of new instances
+for instrumented_class in instrumented.get('new', {}):
+self._register_instance_listener(instrumented_class)
--- End diff --

add documentation explaining what is what


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

[jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006405#comment-16006405
 ] 

ASF GitHub Bot commented on ARIA-213:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115983463
  
--- Diff: aria/storage/instrumentation.py ---
@@ -53,30 +59,76 @@ def track_changes(instrumented=None):
 :param instrumented: A dict from model columns to their python native 
type
 :return: The instrumentation context
 """
-return _Instrumentation(instrumented or _INSTRUMENTED)
+return _Instrumentation(ctx, instrumented or _INSTRUMENTED)
 
 
 class _Instrumentation(object):
 
-def __init__(self, instrumented):
+def __init__(self, ctx, instrumented):
 self.tracked_changes = {}
+self.new_instances = {}
 self.listeners = []
+self._new_modeled_instances = []
+self._ctx = ctx
 self._track_changes(instrumented)
+self._new_instance_index = 0
--- End diff --

remove


> Sporadic tests failures over locked database issue
> --
>
> Key: ARIA-213
> URL: https://issues.apache.org/jira/browse/ARIA-213
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Unit tests sporadically fail over an issue of locked database.
> Here are two stack traces for the same issue that I found on a single run of 
> all of the tests:
> {code}
> _ TestLocalRunScript.test_retry_with_interval 
> __
> self =  object at 0x7f27682af910>
> executor =  object at 0x7f2768250d90>
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0')
> def test_retry_with_interval(self, executor, workflow_context, tmpdir):
> script_path = self._create_script(
> tmpdir,
> linux_script='''#! /bin/bash -e
> ctx task retry retry-message @100
> ''',
> windows_script='''
> ctx task retry retry-message @100
> ''')
> exception = self._run_and_get_task_exception(
> executor, workflow_context,
> >   script_path=script_path)
> tests/orchestrator/execution_plugin/test_local.py:323: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/execution_plugin/test_local.py:452: in 
> _run_and_get_task_exception
> self._run(*args, **kwargs)
> tests/orchestrator/execution_plugin/test_local.py:503: in _run
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:60: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:97: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter
> self._workflow_context.model.task.refresh(task.model_task)
> aria/orchestrator/workflows/core/task.py:191: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ 

[jira] [Commented] (ARIA-157) CLI unit tests for storing service-templates fail on appveyor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006264#comment-16006264
 ] 

ASF GitHub Bot commented on ARIA-157:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/131


> CLI unit tests for storing service-templates fail on appveyor
> -
>
> Key: ARIA-157
> URL: https://issues.apache.org/jira/browse/ARIA-157
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Avia Efrat
>Assignee: Avia Efrat
>
> The failing tests are:
> test_store_no_exception
> test_store_raises_exception_resulting_from_name_uniqueness
> test_store_raises_exception
> They pass on Travis.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006109#comment-16006109
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115935686
  
--- Diff: aria/cli/core/aria.py ---
@@ -65,15 +58,33 @@ def __init__(self, *args, **kwargs):
 super(MutuallyExclusiveOption, self).__init__(*args, **kwargs)
 
 def handle_parse_result(self, ctx, opts, args):
-if self.mutually_exclusive.intersection(opts) and self.name in 
opts:
+if (self.name in opts) and 
self.mutually_exclusive.keys().intersection(opts):
 raise click.UsageError(
 'Illegal usage: `{0}` is mutually exclusive with '
 'arguments: [{1}] ({2}).'.format(
 self.name,
 self.mutuality_string,
 self.mutuality_error_message))
-return super(MutuallyExclusiveOption, self).handle_parse_result(
-ctx, opts, args)
+return super(MutuallyExclusiveOption, 
self).handle_parse_result(ctx, opts, args)
+
+
+def mutually_exclusive_option(*param_decls, **attrs):
+"""
+Makes options mutually exclusive. The decorator must pass a a 
``mutually_exclusive`` argument
+with a list of argument names with which the option is mutually 
exclusive.
+
+NOTE: All mutually exclusive options must use this. It's not enough to 
use it in just one of the
+options.
+"""
+def decorator(func):
+if 'help' in attrs:
--- End diff --

could you please document this func a bit more?
I'm not sure what are `param_decls` and `attrs` in this case, why `help` is 
being inspected, and why the internal `__click_params__` is accessed directly 
here.


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006110#comment-16006110
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115934893
  
--- Diff: aria/cli/core/aria.py ---
@@ -40,16 +41,8 @@
 
 
 class MutuallyExclusiveOption(click.Option):
-"""Makes options mutually exclusive. The option must pass a `cls` 
argument
-with this class name and a `mutually_exclusive` argument with a list of
-argument names it is mutually exclusive with.
-
-NOTE: All mutually exclusive options must use this. It's not enough to
-use it in just one of the options.
-"""
-
 def __init__(self, *args, **kwargs):
-self.mutually_exclusive = set(kwargs.pop('mutually_exclusive', []))
+self.mutually_exclusive = kwargs.pop('mutually_exclusive', tuple())
--- End diff --

why change this from a set to another type?


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006113#comment-16006113
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115935090
  
--- Diff: aria/cli/core/aria.py ---
@@ -65,15 +58,33 @@ def __init__(self, *args, **kwargs):
 super(MutuallyExclusiveOption, self).__init__(*args, **kwargs)
 
 def handle_parse_result(self, ctx, opts, args):
-if self.mutually_exclusive.intersection(opts) and self.name in 
opts:
+if (self.name in opts) and 
self.mutually_exclusive.keys().intersection(opts):
 raise click.UsageError(
 'Illegal usage: `{0}` is mutually exclusive with '
 'arguments: [{1}] ({2}).'.format(
 self.name,
 self.mutuality_string,
 self.mutuality_error_message))
-return super(MutuallyExclusiveOption, self).handle_parse_result(
-ctx, opts, args)
+return super(MutuallyExclusiveOption, 
self).handle_parse_result(ctx, opts, args)
+
+
+def mutually_exclusive_option(*param_decls, **attrs):
+"""
+Makes options mutually exclusive. The decorator must pass a a 
``mutually_exclusive`` argument
--- End diff --

a a a a a


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006111#comment-16006111
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115934967
  
--- Diff: aria/cli/core/aria.py ---
@@ -65,15 +58,33 @@ def __init__(self, *args, **kwargs):
 super(MutuallyExclusiveOption, self).__init__(*args, **kwargs)
 
 def handle_parse_result(self, ctx, opts, args):
-if self.mutually_exclusive.intersection(opts) and self.name in 
opts:
+if (self.name in opts) and 
self.mutually_exclusive.keys().intersection(opts):
--- End diff --

seems like `mutually_exclusive` might be a `tuple` now, so `.keys()` would 
raise an error here.


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006112#comment-16006112
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115934648
  
--- Diff: aria/cli/core/aria.py ---
@@ -325,6 +339,46 @@ def __init__(self):
 default=defaults.SERVICE_TEMPLATE_FILENAME,
 help=helptexts.SERVICE_TEMPLATE_FILENAME)
 
+self.show_full = click.option(
+'-f',
+'--full',
+'mode',
+is_flag=True,
+flag_value='full',
--- End diff --

I think I don't like this approach as much at the previous one.
It seems like it simplifies the code a bit but makes things somewhat more 
confusing to the user - specifically the fact that only the last flag of a 
given "destination" (e.g. `mode`) takes hold.

If instead you have different destinations for these flags, it'd be easier 
to inform the user of misuse (even if for some reason the mutually-exclusive 
mechanism is problematic to use, you can always just manually validate the 
parameters in the command function itself)


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004622#comment-16004622
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115729149
  
--- Diff: aria/cli/commands/service_templates.py ---
@@ -43,32 +45,51 @@ def service_templates():
 @aria.argument('service-template-name')
 @aria.options.verbose()
 @aria.pass_model_storage
+@aria.options.show_all
+@aria.options.show_types
+@aria.options.show_json
+@aria.options.show_yaml
 @aria.pass_logger
-def show(service_template_name, model_storage, logger):
-"""Show information for a specific service templates
+def show(service_template_name, model_storage, all, types, json, yaml, 
logger):
+"""Show information for a specific service template
 
 `SERVICE_TEMPLATE_NAME` is the name of the service template to show 
information on.
 """
-logger.info('Showing service template 
{0}...'.format(service_template_name))
 service_template = 
model_storage.service_template.get_by_name(service_template_name)
-service_template_dict = service_template.to_dict()
-service_template_dict['#services'] = len(service_template.services)
 
-column_formatters = \
-
dict(description=table.trim_formatter_generator(DESCRIPTION_FIELD_LENGTH_LIMIT))
-columns = SERVICE_TEMPLATE_COLUMNS + ['#services']
-table.print_data(columns, service_template_dict, 'Service-template:',
- column_formatters=column_formatters, col_max_width=50)
-
-if service_template_dict['description'] is not None:
-logger.info('Description:')
-
logger.info('{0}{1}'.format(service_template_dict['description'].encode('UTF-8')
 or '',
-os.linesep))
-
-if service_template.services:
-logger.info('Existing services:')
-for service in service_template.services:
-logger.info('\t{0}'.format(service.name))
+if json or yaml:
--- End diff --

i'm a bit confused by this. so `yaml`, `json`, and `all` all mean all,
and `json` and `yaml` can be used simulatniously but only `json` will get 
printed.

Re the latter problem, please have a look at `aria.py` for how to make 
flags mutually-exclusive.
re the former, it might be better to also have the `all` flag as 
mutually-exclusive with the other two..


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004603#comment-16004603
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115726284
  
--- Diff: aria/cli/table.py ---
@@ -85,7 +85,7 @@ def get_values_per_column(column, row_data):
 
 return val
 else:
-return defaults[column]
+return defaults.get(column) if defaults is not None else None
--- End diff --

should instead have a `defaults = defaults or {}` before calling the nested 
method


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Enhance CLI "show" commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004602#comment-16004602
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r115726390
  
--- Diff: aria/cli/commands/services.py ---
@@ -38,6 +41,53 @@ def services():
 pass
 
 
+@services.command(name='show',
+  short_help='Display service information')
+@aria.argument('service-name')
+@aria.options.verbose()
+@aria.options.show_all
+@aria.options.show_graph
+@aria.options.show_json
+@aria.options.show_yaml
+@aria.pass_model_storage
+@aria.pass_logger
+def show(service_name, model_storage, all, graph, json, yaml, logger):
+"""Show information for a specific service template
+
+`SERVICE_NAME` is the name of the service to display information on.
+"""
+logger.info('Showing service {0}...'.format(service_name))
+service = model_storage.service.get_by_name(service_name)
+
+if json or yaml:
+all = True
--- End diff --

i'd rename the flag, it's both unclear and a python keyword. `show_all` or 
`all_info` might be better.


> Enhance CLI "show" commands
> ---
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> For {{aria service-templates show}} and {{aria services show}} add support 
> for dumping, including graphs and type hierarchies, in JSON and YAML.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-62) PyLint should cover "extensions", not just "aria"

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004369#comment-16004369
 ] 

ASF GitHub Bot commented on ARIA-62:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/130


> PyLint should cover "extensions", not just "aria"
> -
>
> Key: ARIA-62
> URL: https://issues.apache.org/jira/browse/ARIA-62
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Ran Ziv
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-62) PyLint should cover "extensions", not just "aria"

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004346#comment-16004346
 ] 

ASF GitHub Bot commented on ARIA-62:


GitHub user ran-z opened a pull request:

https://github.com/apache/incubator-ariatosca/pull/130

ARIA-62 Apply pylint on extensions



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/incubator-ariatosca 
ARIA-62-apply-pylint-on-extensions

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-ariatosca/pull/130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #130


commit 0af9e638b7c75eea6eac39374b7176f9dc2c645a
Author: Ran Ziv 
Date:   2017-05-10T09:08:01Z

ARIA-62 Apply pylint on extensions




> PyLint should cover "extensions", not just "aria"
> -
>
> Key: ARIA-62
> URL: https://issues.apache.org/jira/browse/ARIA-62
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003484#comment-16003484
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on the issue:

https://github.com/apache/incubator-ariatosca/pull/100
  
@tliron and I discussed his replies via zoom, and this pull request is free 
to merge (with minor changes).


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002780#comment-16002780
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/128


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002766#comment-16002766
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115502906
  
--- Diff: tests/cli/test_service_templates.py ---
@@ -244,3 +258,13 @@ def test_create_archive_successful(self, monkeypatch, 
mock_object):
 monkeypatch.setattr(csar, 'write', mock_object)
 self.invoke('service_templates create_archive stubpath stubdest')
 assert 'CSAR archive created at stubdest' in 
self.logger_output_string
+
+def test_create_archive_from_relative_path(self, monkeypatch, 
mock_object):
+
+monkeypatch.setattr(os.path, 'isfile', lambda x: True)
+monkeypatch.setattr(os.path, 'isfile', mock_object)
--- End diff --

Indeed. So I removed that line.


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002756#comment-16002756
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115502106
  
--- Diff: tests/cli/test_service_templates.py ---
@@ -131,6 +133,18 @@ def test_store_no_exception(self, monkeypatch, 
mock_object):
 assert 'Service template {name} stored'.format(
 name=mock_models.SERVICE_TEMPLATE_NAME) in 
self.logger_output_string
 
+def test_store_relative_path_single_yaml_file(self, monkeypatch, 
mock_object):
+monkeypatch.setattr(Core, 'create_service_template', mock_object)
+monkeypatch.setattr(os.path, 'isfile', lambda x: True)
+monkeypatch.setattr(service_template_utils, '_is_archive', lambda 
x: False)
+
+self.invoke('service_templates store service_template.yaml 
{name}'.format(
+name=mock_models.SERVICE_TEMPLATE_NAME))
+
+mock_object.assert_called_with(os.path.join(os.getcwd(), 
'service_template.yaml'),
--- End diff --

Nothing serious, just thought that it demonstrates the fact that we are 
expecting the paths to be relative to the current working directory.


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002753#comment-16002753
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115501254
  
--- Diff: aria/cli/service_template_utils.py ---
@@ -53,7 +53,7 @@ def get(source, service_template_filename):
 return _get_service_template_file_from_archive(source, 
service_template_filename)
--- End diff --

Added tests.


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002744#comment-16002744
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115498975
  
--- Diff: aria/cli/csar.py ---
@@ -22,7 +22,7 @@
 import requests
 from ruamel import yaml
 
-
+CSAR_FILE_EXTENSION = '.csar'
--- End diff --

do a quick search for `.csar`, there should be at least one more reference 
to it in modules you've touched anyway :)


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002742#comment-16002742
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115498822
  
--- Diff: tests/cli/test_service_templates.py ---
@@ -244,3 +258,13 @@ def test_create_archive_successful(self, monkeypatch, 
mock_object):
 monkeypatch.setattr(csar, 'write', mock_object)
 self.invoke('service_templates create_archive stubpath stubdest')
 assert 'CSAR archive created at stubdest' in 
self.logger_output_string
+
+def test_create_archive_from_relative_path(self, monkeypatch, 
mock_object):
+
+monkeypatch.setattr(os.path, 'isfile', lambda x: True)
+monkeypatch.setattr(os.path, 'isfile', mock_object)
--- End diff --

doesnt this line override the previous one?


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002743#comment-16002743
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115499156
  
--- Diff: tests/cli/test_service_templates.py ---
@@ -131,6 +133,18 @@ def test_store_no_exception(self, monkeypatch, 
mock_object):
 assert 'Service template {name} stored'.format(
 name=mock_models.SERVICE_TEMPLATE_NAME) in 
self.logger_output_string
 
+def test_store_relative_path_single_yaml_file(self, monkeypatch, 
mock_object):
+monkeypatch.setattr(Core, 'create_service_template', mock_object)
+monkeypatch.setattr(os.path, 'isfile', lambda x: True)
+monkeypatch.setattr(service_template_utils, '_is_archive', lambda 
x: False)
+
+self.invoke('service_templates store service_template.yaml 
{name}'.format(
+name=mock_models.SERVICE_TEMPLATE_NAME))
+
+mock_object.assert_called_with(os.path.join(os.getcwd(), 
'service_template.yaml'),
--- End diff --

just curious why you've decided to use `join` and `getcwd` here rather than 
absolute path (like you did in the actual code). not that it should matter :)


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-160) Operation toolbelt unit tests fail spordically

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002497#comment-16002497
 ] 

ASF GitHub Bot commented on ARIA-160:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/126


> Operation toolbelt unit tests fail spordically
> --
>
> Key: ARIA-160
> URL: https://issues.apache.org/jira/browse/ARIA-160
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> The unit tests in the module {{test_toolbelt.py}} fail remote builds 
> (TravisCI, Jenkins) sporadically.
> Stack trace:
> {code}
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> executor =  at 0x7f57f8482a50>
> def test_host_ip(workflow_context, executor):
> interface_name = 'Standard'
> operation_name = 'create'
> _, dependency_node, _, _, _ = _get_elements(workflow_context)
> inputs = {'putput': True}
> interface = mock.models.create_interface(
> dependency_node.service,
> interface_name=interface_name,
> operation_name=operation_name,
> operation_kwargs=dict(implementation=op_path(host_ip, 
> module_path=__name__),
>   inputs=inputs)
> )
> dependency_node.interfaces[interface.name] = interface
> workflow_context.model.node.update(dependency_node)
> 
> @workflow
> def basic_workflow(graph, **_):
> graph.add_tasks(
> api.task.OperationTask.for_node(
> node=dependency_node,
> interface_name=interface_name,
> operation_name=operation_name,
> inputs=inputs
> )
> )
> 
> >   execute(workflow_func=basic_workflow, 
> > workflow_context=workflow_context, executor=executor)
> tests/orchestrator/context/test_toolbelt.py:101: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/context/__init__.py:29: in execute
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:59: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:96: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/task.py:259: in __getattr__
> return getattr(self.model_task, attr)
> aria/orchestrator/workflows/core/task.py:178: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f57f97ee250>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t..._fk AS 
> task_execution_fk, task.relationship_fk AS task_relationship_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f57f83b1810>
> def do_execute(self, cursor, statement, parameters, context=None):
> >   cursor.execute(statement, parameters)
> E   InterfaceError: (sqlite3.InterfaceError) 

[jira] [Commented] (ARIA-165) Make node name suffix UUIDs become more readable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002300#comment-16002300
 ] 

ASF GitHub Bot commented on ARIA-165:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/125


> Make node name suffix UUIDs become more readable
> 
>
> Key: ARIA-165
> URL: https://issues.apache.org/jira/browse/ARIA-165
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> Node names are currently constructed of the node template name with a UUID 
> suffix. While this makes the node name unique across all services, it also 
> makes the node names less readable, and since nodes are already unique via 
> their ID, it might be better to use a different type of suffix.
> One option would be to generate a shorter, no capital letters string, which 
> would be more readable, and (in very high probability) unique in a single 
> service.
> Another option would be to go for a running number suffix for each node of a 
> given node template, which could be simple, unique and readable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-165) Make node name suffix UUIDs become more readable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002268#comment-16002268
 ] 

ASF GitHub Bot commented on ARIA-165:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/125#discussion_r115431430
  
--- Diff: aria/modeling/service_template.py ---
@@ -549,9 +549,14 @@ def as_raw(self):
 ('requirement_templates', 
formatting.as_raw_list(self.requirement_templates
 
 def instantiate(self, container):
-context = ConsumptionContext.get_thread_local()
 from . import models
-name = context.modeling.generate_node_id(self.name)
+import pydevd; pydevd.settrace('localhost', suspend=False)
+if self.nodes:
+highest_name_suffix = max(self.nodes, key=lambda n: 
n.name).name.rsplit('_', 1)[1]
--- End diff --

im not sure this would work;
note that `mynode_2` will come after `mynode_11` :|


> Make node name suffix UUIDs become more readable
> 
>
> Key: ARIA-165
> URL: https://issues.apache.org/jira/browse/ARIA-165
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Node names are currently constructed of the node template name with a UUID 
> suffix. While this makes the node name unique across all services, it also 
> makes the node names less readable, and since nodes are already unique via 
> their ID, it might be better to use a different type of suffix.
> One option would be to generate a shorter, no capital letters string, which 
> would be more readable, and (in very high probability) unique in a single 
> service.
> Another option would be to go for a running number suffix for each node of a 
> given node template, which could be simple, unique and readable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-165) Make node name suffix UUIDs become more readable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002266#comment-16002266
 ] 

ASF GitHub Bot commented on ARIA-165:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/125#discussion_r115431168
  
--- Diff: aria/modeling/service_template.py ---
@@ -549,9 +549,14 @@ def as_raw(self):
 ('requirement_templates', 
formatting.as_raw_list(self.requirement_templates
 
 def instantiate(self, container):
-context = ConsumptionContext.get_thread_local()
 from . import models
-name = context.modeling.generate_node_id(self.name)
+import pydevd; pydevd.settrace('localhost', suspend=False)
--- End diff --

poo


> Make node name suffix UUIDs become more readable
> 
>
> Key: ARIA-165
> URL: https://issues.apache.org/jira/browse/ARIA-165
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Node names are currently constructed of the node template name with a UUID 
> suffix. While this makes the node name unique across all services, it also 
> makes the node names less readable, and since nodes are already unique via 
> their ID, it might be better to use a different type of suffix.
> One option would be to generate a shorter, no capital letters string, which 
> would be more readable, and (in very high probability) unique in a single 
> service.
> Another option would be to go for a running number suffix for each node of a 
> given node template, which could be simple, unique and readable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-160) Operation toolbelt unit tests fail spordically

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002241#comment-16002241
 ] 

ASF GitHub Bot commented on ARIA-160:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/126#discussion_r115428189
  
--- Diff: tests/helpers.py ---
@@ -34,16 +33,17 @@ def get_service_template_uri(*args):
 
 
 class FilesystemDataHolder(object):
-_tmpfile = tempfile.NamedTemporaryFile('w')
 
 def _load(self):
-return json.load(open(self._tmpfile.name))
+return json.load(open(self._path))
 
 def _dump(self, value):
-return json.dump(value, open(self._tmpfile.name, 'w'))
+return json.dump(value, open(self._path, 'w'))
 
-def __init__(self):
-self.clear()
+def __init__(self, path, reset=False):
+self._path = path
+if reset or not os.path.exists(self._path) or 
open(self._path).read() == '':
--- End diff --

no closing of the file?


> Operation toolbelt unit tests fail spordically
> --
>
> Key: ARIA-160
> URL: https://issues.apache.org/jira/browse/ARIA-160
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> The unit tests in the module {{test_toolbelt.py}} fail remote builds 
> (TravisCI, Jenkins) sporadically.
> Stack trace:
> {code}
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> executor =  at 0x7f57f8482a50>
> def test_host_ip(workflow_context, executor):
> interface_name = 'Standard'
> operation_name = 'create'
> _, dependency_node, _, _, _ = _get_elements(workflow_context)
> inputs = {'putput': True}
> interface = mock.models.create_interface(
> dependency_node.service,
> interface_name=interface_name,
> operation_name=operation_name,
> operation_kwargs=dict(implementation=op_path(host_ip, 
> module_path=__name__),
>   inputs=inputs)
> )
> dependency_node.interfaces[interface.name] = interface
> workflow_context.model.node.update(dependency_node)
> 
> @workflow
> def basic_workflow(graph, **_):
> graph.add_tasks(
> api.task.OperationTask.for_node(
> node=dependency_node,
> interface_name=interface_name,
> operation_name=operation_name,
> inputs=inputs
> )
> )
> 
> >   execute(workflow_func=basic_workflow, 
> > workflow_context=workflow_context, executor=executor)
> tests/orchestrator/context/test_toolbelt.py:101: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/context/__init__.py:29: in execute
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:59: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:96: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/task.py:259: in __getattr__
> return getattr(self.model_task, attr)
> aria/orchestrator/workflows/core/task.py:178: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> 

[jira] [Commented] (ARIA-160) Operation toolbelt unit tests fail spordically

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002242#comment-16002242
 ] 

ASF GitHub Bot commented on ARIA-160:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/126#discussion_r115427438
  
--- Diff: tests/helpers.py ---
@@ -34,16 +33,17 @@ def get_service_template_uri(*args):
 
 
 class FilesystemDataHolder(object):
-_tmpfile = tempfile.NamedTemporaryFile('w')
 
 def _load(self):
-return json.load(open(self._tmpfile.name))
+return json.load(open(self._path))
 
 def _dump(self, value):
-return json.dump(value, open(self._tmpfile.name, 'w'))
+return json.dump(value, open(self._path, 'w'))
 
-def __init__(self):
-self.clear()
+def __init__(self, path, reset=False):
--- End diff --

move init to be the first method


> Operation toolbelt unit tests fail spordically
> --
>
> Key: ARIA-160
> URL: https://issues.apache.org/jira/browse/ARIA-160
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> The unit tests in the module {{test_toolbelt.py}} fail remote builds 
> (TravisCI, Jenkins) sporadically.
> Stack trace:
> {code}
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> executor =  at 0x7f57f8482a50>
> def test_host_ip(workflow_context, executor):
> interface_name = 'Standard'
> operation_name = 'create'
> _, dependency_node, _, _, _ = _get_elements(workflow_context)
> inputs = {'putput': True}
> interface = mock.models.create_interface(
> dependency_node.service,
> interface_name=interface_name,
> operation_name=operation_name,
> operation_kwargs=dict(implementation=op_path(host_ip, 
> module_path=__name__),
>   inputs=inputs)
> )
> dependency_node.interfaces[interface.name] = interface
> workflow_context.model.node.update(dependency_node)
> 
> @workflow
> def basic_workflow(graph, **_):
> graph.add_tasks(
> api.task.OperationTask.for_node(
> node=dependency_node,
> interface_name=interface_name,
> operation_name=operation_name,
> inputs=inputs
> )
> )
> 
> >   execute(workflow_func=basic_workflow, 
> > workflow_context=workflow_context, executor=executor)
> tests/orchestrator/context/test_toolbelt.py:101: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/context/__init__.py:29: in execute
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:59: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:96: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/task.py:259: in __getattr__
> return getattr(self.model_task, attr)
> aria/orchestrator/workflows/core/task.py:178: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> 

[jira] [Commented] (ARIA-160) Operation toolbelt unit tests fail spordically

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002240#comment-16002240
 ] 

ASF GitHub Bot commented on ARIA-160:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/126#discussion_r115428827
  
--- Diff: tests/orchestrator/context/test_operation.py ---
@@ -97,31 +103,31 @@ def basic_workflow(graph, **_):
 
 execute(workflow_func=basic_workflow, workflow_context=ctx, 
executor=thread_executor)
 
-assert global_test_holder['ctx_name'] == 
context.operation.NodeOperationContext.__name__
+assert test_holder['ctx_name'] == 
context.operation.NodeOperationContext.__name__
--- End diff --

TBH i'd called it `dataholder` everywhere it says simply `holder` but up to 
you


> Operation toolbelt unit tests fail spordically
> --
>
> Key: ARIA-160
> URL: https://issues.apache.org/jira/browse/ARIA-160
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
> Fix For: 0.1.0
>
>
> The unit tests in the module {{test_toolbelt.py}} fail remote builds 
> (TravisCI, Jenkins) sporadically.
> Stack trace:
> {code}
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> executor =  at 0x7f57f8482a50>
> def test_host_ip(workflow_context, executor):
> interface_name = 'Standard'
> operation_name = 'create'
> _, dependency_node, _, _, _ = _get_elements(workflow_context)
> inputs = {'putput': True}
> interface = mock.models.create_interface(
> dependency_node.service,
> interface_name=interface_name,
> operation_name=operation_name,
> operation_kwargs=dict(implementation=op_path(host_ip, 
> module_path=__name__),
>   inputs=inputs)
> )
> dependency_node.interfaces[interface.name] = interface
> workflow_context.model.node.update(dependency_node)
> 
> @workflow
> def basic_workflow(graph, **_):
> graph.add_tasks(
> api.task.OperationTask.for_node(
> node=dependency_node,
> interface_name=interface_name,
> operation_name=operation_name,
> inputs=inputs
> )
> )
> 
> >   execute(workflow_func=basic_workflow, 
> > workflow_context=workflow_context, executor=executor)
> tests/orchestrator/context/test_toolbelt.py:101: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/context/__init__.py:29: in execute
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:59: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:96: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/task.py:259: in __getattr__
> return getattr(self.model_task, attr)
> aria/orchestrator/workflows/core/task.py:178: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

[jira] [Commented] (ARIA-165) Make node name suffix UUIDs become more readable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002231#comment-16002231
 ] 

ASF GitHub Bot commented on ARIA-165:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/125#discussion_r115427582
  
--- Diff: aria/modeling/service_template.py ---
@@ -549,9 +549,13 @@ def as_raw(self):
 ('requirement_templates', 
formatting.as_raw_list(self.requirement_templates
 
 def instantiate(self, container):
-context = ConsumptionContext.get_thread_local()
 from . import models
-name = context.modeling.generate_node_id(self.name)
+if self.nodes:
+latest_node_index = self.nodes[-1].name.rsplit('_', 1)[1]
--- End diff --

Pretty sure sql orders according to the indexed column


> Make node name suffix UUIDs become more readable
> 
>
> Key: ARIA-165
> URL: https://issues.apache.org/jira/browse/ARIA-165
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Node names are currently constructed of the node template name with a UUID 
> suffix. While this makes the node name unique across all services, it also 
> makes the node names less readable, and since nodes are already unique via 
> their ID, it might be better to use a different type of suffix.
> One option would be to generate a shorter, no capital letters string, which 
> would be more readable, and (in very high probability) unique in a single 
> service.
> Another option would be to go for a running number suffix for each node of a 
> given node template, which could be simple, unique and readable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-165) Make node name suffix UUIDs become more readable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002230#comment-16002230
 ] 

ASF GitHub Bot commented on ARIA-165:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/125#discussion_r115427157
  
--- Diff: aria/modeling/service_template.py ---
@@ -549,9 +549,13 @@ def as_raw(self):
 ('requirement_templates', 
formatting.as_raw_list(self.requirement_templates
 
 def instantiate(self, container):
-context = ConsumptionContext.get_thread_local()
 from . import models
-name = context.modeling.generate_node_id(self.name)
+if self.nodes:
+latest_node_index = self.nodes[-1].name.rsplit('_', 1)[1]
--- End diff --

is the nodes list guaranteed to be ordered?
if not then we need to find the max

either way, i'd change `latest` to `highest`


> Make node name suffix UUIDs become more readable
> 
>
> Key: ARIA-165
> URL: https://issues.apache.org/jira/browse/ARIA-165
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Node names are currently constructed of the node template name with a UUID 
> suffix. While this makes the node name unique across all services, it also 
> makes the node names less readable, and since nodes are already unique via 
> their ID, it might be better to use a different type of suffix.
> One option would be to generate a shorter, no capital letters string, which 
> would be more readable, and (in very high probability) unique in a single 
> service.
> Another option would be to go for a running number suffix for each node of a 
> given node template, which could be simple, unique and readable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-165) Make node name suffix UUIDs become more readable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002227#comment-16002227
 ] 

ASF GitHub Bot commented on ARIA-165:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/125#discussion_r115426507
  
--- Diff: aria/modeling/service_template.py ---
@@ -549,9 +549,13 @@ def as_raw(self):
 ('requirement_templates', 
formatting.as_raw_list(self.requirement_templates
 
 def instantiate(self, container):
-context = ConsumptionContext.get_thread_local()
 from . import models
-name = context.modeling.generate_node_id(self.name)
+if self.nodes:
+latest_node_index = self.nodes[-1].name.rsplit('_', 1)[1]
+index = int(latest_node_index) + 1
+else:
+index = 0
--- End diff --

i'd start from index=1 actually


> Make node name suffix UUIDs become more readable
> 
>
> Key: ARIA-165
> URL: https://issues.apache.org/jira/browse/ARIA-165
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> Node names are currently constructed of the node template name with a UUID 
> suffix. While this makes the node name unique across all services, it also 
> makes the node names less readable, and since nodes are already unique via 
> their ID, it might be better to use a different type of suffix.
> One option would be to generate a shorter, no capital letters string, which 
> would be more readable, and (in very high probability) unique in a single 
> service.
> Another option would be to go for a running number suffix for each node of a 
> given node template, which could be simple, unique and readable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-230) Dry execution doesn't log empty operations

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002022#comment-16002022
 ] 

ASF GitHub Bot commented on ARIA-230:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/127


> Dry execution doesn't log empty operations
> --
>
> Key: ARIA-230
> URL: https://issues.apache.org/jira/browse/ARIA-230
> Project: AriaTosca
>  Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Dry execution doesn't log empty operations



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001336#comment-16001336
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115324450
  
--- Diff: aria/cli/commands/service_templates.py ---
@@ -195,7 +195,9 @@ def create_archive(service_template_path, destination, 
logger):
 `destination` is the path of the output CSAR archive file
 """
 logger.info('Creating a CSAR archive')
-csar.write(os.path.dirname(service_template_path), 
service_template_path, destination, logger)
+if not destination.endswith('.csar'):
--- End diff --

maybe extract `.csar` suffix to a constant in csar.py? seems like its 
defined in several places now


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001338#comment-16001338
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115325816
  
--- Diff: aria/cli/service_template_utils.py ---
@@ -53,7 +53,7 @@ def get(source, service_template_filename):
 return _get_service_template_file_from_archive(source, 
service_template_filename)
--- End diff --

What about adding a short test or two for these scenarios? :)


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001337#comment-16001337
 ] 

ASF GitHub Bot commented on ARIA-210:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/128#discussion_r115324345
  
--- Diff: aria/cli/csar.py ---
@@ -38,17 +38,19 @@
 }
 
 
-def write(source, entry, destination, logger):
-source = os.path.expanduser(source)
-destination = os.path.expanduser(destination)
-entry_definitions = os.path.join(source, entry)
+def write(service_template_path, destination, logger):
--- End diff --

good, i like the signature better this way


> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-210) Relative paths may cause issues in `aria service-templates` commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001271#comment-16001271
 ] 

ASF GitHub Bot commented on ARIA-210:
-

GitHub user AviaE opened a pull request:

https://github.com/apache/incubator-ariatosca/pull/128

ARIA-210 Handle relative paths in CLI service-templates

This was a rather simple change, mainly involving adding absolute path
references.

The problems were only in `service-templates store` and in
`service-templates create-archive`.
`service-templates validate` was not affected.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/incubator-ariatosca 
ARIA-210-handle-relative-paths-in-cli-service-templates

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-ariatosca/pull/128.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #128


commit 0a3ea7991d6218438318d93232f82e7d7d42d4ae
Author: Avia Efrat 
Date:   2017-05-08T14:45:23Z

ARIA-210 Handle relative paths in CLI service-templates

This was a rather simple change, mainly involving adding absolute path
references.

The problems were only in `service-templates store` and in
`service-templates create-archive`.
`service-templates validate` was not affected.




> Relative paths may cause issues in `aria service-templates` commands
> 
>
> Key: ARIA-210
> URL: https://issues.apache.org/jira/browse/ARIA-210
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Avia Efrat
>
> The CLI commands {{aria service-templates store}} and {{aria 
> service-templates create-archive}} use the {{os.path.dirname}} method to get 
> the directory of the main service-template file.
> In the former ({{store}} command) this causes errors when the user is already 
> in the directory of the main service-template file and passes the file 
> directly (e.g. {{aria service-templates store service-template.yaml 
> my-service-template}})) - It'll fail when trying to copy the 
> service-template's resources like so:
> {{cannot copy tree '': not a directory}}
> In the latter ({{create-archive}}), any relative path usage will result in an 
> error claiming the file does not exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001019#comment-16001019
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115289497
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
--- End diff --

Because "Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" > "1.1-beta1". The qualifier is a float that is an 
extra, but if the qualifier is not present "infinity" should always win over 
it. It shouldn't matter what's "more natural" seeming: this is an underscored 
internal var that's never seen by users of the lib. Users instead use 
`VersionString.NULL` or can init `VersionString` with None.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001012#comment-16001012
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115289271
  
--- Diff: aria/modeling/service_template.py ---
@@ -2131,13 +2132,15 @@ def resolve(self, model_storage):
 # moved to.
 plugins = model_storage.plugin.list()
 matching_plugins = []
-for plugin in plugins:
-# TODO: we need to use a version comparator
-if (plugin.name == self.name) and \
-((self.version is None) or (plugin.package_version >= 
self.version)):
-matching_plugins.append(plugin)
+if plugins:
+for plugin in plugins:
+if (plugin.name == self.name) and \
--- End diff --

I strongly disagree, especially in this case when there is `or` and `and` 
in the same clause. Do you otherwise expect the reader to know the priority of 
logical binary operations? Parentheses remove any potential confusion.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001014#comment-16001014
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on the issue:

https://github.com/apache/incubator-ariatosca/pull/100
  
Responses.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001016#comment-16001016
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115289404
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
--- End diff --

This is a user-exposed function and None values do occur in the wild.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001018#comment-16001018
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115289426
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
--- End diff --

It is true, there is no tests for this function specifically with a None 
value. I will add it.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001011#comment-16001011
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115288925
  
--- Diff: aria/modeling/service_template.py ---
@@ -33,7 +33,8 @@
 from ..parser import validation
 from ..parser.consumption import ConsumptionContext
 from ..parser.reading import deepcopy_with_locators
-from ..utils import collections, formatting, console
+from ..utils import (collections, formatting, console)
--- End diff --

We have discussed this and have agreed that it can be in one line if it 
fits. We have a lot of code that is like this.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001010#comment-16001010
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115288664
  
--- Diff: aria/modeling/service_template.py ---
@@ -33,7 +33,8 @@
 from ..parser import validation
--- End diff --

I don't understand. What does this have to do with this line in the code? 
Where in the commit message? We have limited our commit messages to one line 
with reference to the JIRA for more information. I know you tend to add more 
info, but the rest of us don't normally.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001007#comment-16001007
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115288142
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
+
+# Split to primary and qualifier on '-'
+split = version.split('-', 2)
+if len(split) == 2:
+primary, qualifier = split
+else:
+primary = split[0]
+qualifier = None
+
+# Parse primary
+split = primary.split('.')
+primary = []
+for element in split:
+if _DIGITS_RE.match(element) is None:
+# Invalid version string
+return _NULL
+try:
+element = int(element)
+except ValueError:
+# Invalid version string
+return _NULL
+primary.append(element)
+
+# Remove redundant zeros
+for element in reversed(primary):
+if element == 0:
+primary.pop()
+else:
+break
+primary = tuple(primary)
+
+# Parse qualifier
+if qualifier is not None:
+if _DIGITS_RE.match(qualifier) is not None:
+# Integer qualifer
+try:
+qualifier = float(int(qualifier))
  

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001006#comment-16001006
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115287946
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
+
+# Split to primary and qualifier on '-'
+split = version.split('-', 2)
+if len(split) == 2:
+primary, qualifier = split
+else:
+primary = split[0]
+qualifier = None
+
+# Parse primary
+split = primary.split('.')
+primary = []
+for element in split:
+if _DIGITS_RE.match(element) is None:
+# Invalid version string
+return _NULL
+try:
--- End diff --

Maybe it's too many digits for the current Python platform's size of int? 
Seems to me that it's best not to assume it would always work. Utility methods 
need to be especially robust.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of 

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000985#comment-16000985
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115285799
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
+
+# Split to primary and qualifier on '-'
+split = version.split('-', 2)
+if len(split) == 2:
+primary, qualifier = split
+else:
--- End diff --

Oops, yes, changed to rsplit on 1.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000958#comment-16000958
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115282963
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
--- End diff --

It is true, there is no test for this function specifically with a None 
value. I will add it.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000954#comment-16000954
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115281803
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
--- End diff --

Because "Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" > "1.1-beta1". The qualifier is a float that is an 
extra, but if the qualifier is not present "infinity" should always win over 
it. It shouldn't matter what's "more natural" seeming: this is an underscored 
internal var that's never seen by users of the lib. Users instead use 
`VersionString.NULL` or can init `VersionString` with None.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000943#comment-16000943
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user tliron commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115281253
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
--- End diff --

Because "Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" > "1.1-beta1". The qualifier is a float that is an 
extra, but if the qualifier is not present "infinity" should always win over 
it. It shouldn't matter what's "more natural" seeming: this is an underscored 
internal var that's never seen by users of the lib.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999893#comment-15999893
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115145852
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
+
+# Split to primary and qualifier on '-'
+split = version.split('-', 2)
+if len(split) == 2:
+primary, qualifier = split
+else:
+primary = split[0]
+qualifier = None
+
+# Parse primary
+split = primary.split('.')
+primary = []
+for element in split:
+if _DIGITS_RE.match(element) is None:
+# Invalid version string
+return _NULL
+try:
+element = int(element)
+except ValueError:
+# Invalid version string
+return _NULL
+primary.append(element)
+
+# Remove redundant zeros
+for element in reversed(primary):
+if element == 0:
+primary.pop()
+else:
+break
+primary = tuple(primary)
+
+# Parse qualifier
+if qualifier is not None:
+if _DIGITS_RE.match(qualifier) is not None:
+# Integer qualifer
+try:
+qualifier = float(int(qualifier))
   

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999885#comment-15999885
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115142636
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
+
+# Split to primary and qualifier on '-'
+split = version.split('-', 2)
+if len(split) == 2:
+primary, qualifier = split
+else:
+primary = split[0]
+qualifier = None
+
+# Parse primary
+split = primary.split('.')
+primary = []
+for element in split:
+if _DIGITS_RE.match(element) is None:
+# Invalid version string
+return _NULL
+try:
+element = int(element)
+except ValueError:
+# Invalid version string
+return _NULL
+primary.append(element)
+
+# Remove redundant zeros
+for element in reversed(primary):
+if element == 0:
+primary.pop()
+else:
+break
+primary = tuple(primary)
+
+# Parse qualifier
+if qualifier is not None:
+if _DIGITS_RE.match(qualifier) is not None:
+# Integer qualifer
+try:
+qualifier = float(int(qualifier))
   

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999895#comment-15999895
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115141664
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
--- End diff --

This line will always be called, since `version` cannot be `None`.
And except the case when `VersionString` is called with no arguments, this 
line was already called in `__init__`.
In addition, in all of your tests, you never pass through line 93.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999890#comment-15999890
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115142886
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
+
+# Split to primary and qualifier on '-'
+split = version.split('-', 2)
+if len(split) == 2:
+primary, qualifier = split
+else:
--- End diff --

The `2` in `split` means you include the part to the right of the second 
separator as the third element in the result.
Which means that if you get passed `1.2-3-beta4`, you will treat it as a 
valid version string with the value `1.2`.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999891#comment-15999891
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115144550
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
--- End diff --

How can version be `None` here?


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999887#comment-15999887
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115141706
  
--- Diff: aria/modeling/service_template.py ---
@@ -2131,13 +2132,15 @@ def resolve(self, model_storage):
 # moved to.
 plugins = model_storage.plugin.list()
 matching_plugins = []
-for plugin in plugins:
-# TODO: we need to use a version comparator
-if (plugin.name == self.name) and \
-((self.version is None) or (plugin.package_version >= 
self.version)):
-matching_plugins.append(plugin)
+if plugins:
+for plugin in plugins:
+if (plugin.name == self.name) and \
--- End diff --

I know that wrapping `if` conditions with parenthesis is your thing, but in 
this case it seems to cause some clutter...


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999892#comment-15999892
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115145936
  
--- Diff: aria/modeling/service_template.py ---
@@ -33,7 +33,8 @@
 from ..parser import validation
--- End diff --

Please expand the commit message by adding information on the purpose of 
your change/addition.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999888#comment-15999888
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115142635
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
+
+_DIGITS_RE = re.compile(r'^\d+$')
+
+_PREFIXES = {
+'dev':   0.0001,
+'alpha': 0.001,
+'beta':  0.01,
+'rc':0.1
+}
+
+
+class VersionString(unicode):
+"""
+Version string that can be compared, sorted, made unique in a set, and 
used as a unique dict
+key.
+
+The primary part of the string is one or more dot-separated natural 
numbers. Trailing zeroes
+are treated as redundant, e.g. "1.0.0" == "1.0" == "1".
+
+An optional qualifier can be added after a "-". The qualifier can be a 
natural number or a
+specially treated prefixed natural number, e.g. "1.1-beta1" > 
"1.1-alpha2". The case of the
+prefix is ignored.
+
+Numeric qualifiers will always be greater than prefixed integer 
qualifiers, e.g. "1.1-1" >
+"1.1-beta1".
+
+Versions without a qualifier will always be greater than their 
equivalents with a qualifier,
+e.g. e.g. "1.1" > "1.1-1".
+
+Any value that does not conform to this format will be treated as a 
zero version, which would
+be lesser than any non-zero version.
+
+For efficient list sorts use the ``key`` property, e.g.:
+``sorted(versions, key=lambda x: x.key)``
+"""
+
+NULL = None # initialized below
+
+def __init__(self, value=None):
+if value is not None:
+super(VersionString, self).__init__(value)
+self.key = parse_version_string(self)
+
+def __eq__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key == version.key
+
+def __lt__(self, version):
+if not isinstance(version, VersionString):
+version = VersionString(version)
+return self.key < version.key
+
+def __hash__(self):
+return self.key.__hash__()
+
+
+def parse_version_string(version): # pylint: disable=too-many-branches
+"""
+Parses a version string.
+
+:param version: The version string
+:returns: The primary tuple and qualifier float
+:rtype: ((int), float)
+"""
+
+if version is None:
+return _NULL
+version = unicode(version)
+
+# Split to primary and qualifier on '-'
+split = version.split('-', 2)
+if len(split) == 2:
+primary, qualifier = split
+else:
+primary = split[0]
+qualifier = None
+
+# Parse primary
+split = primary.split('.')
+primary = []
+for element in split:
+if _DIGITS_RE.match(element) is None:
+# Invalid version string
+return _NULL
+try:
+element = int(element)
+except ValueError:
+# Invalid version string
+return _NULL
+primary.append(element)
+
+# Remove redundant zeros
+for element in reversed(primary):
+if element == 0:
+primary.pop()
+else:
+break
+primary = tuple(primary)
+
+# Parse qualifier
+if qualifier is not None:
+if _DIGITS_RE.match(qualifier) is not None:
+# Integer qualifer
+try:
--- End diff --

Same as before, what can be 

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999894#comment-15999894
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115141677
  
--- Diff: aria/modeling/service_template.py ---
@@ -33,7 +33,8 @@
 from ..parser import validation
 from ..parser.consumption import ConsumptionContext
 from ..parser.reading import deepcopy_with_locators
-from ..utils import collections, formatting, console
+from ..utils import (collections, formatting, console)
--- End diff --

According to 
https://cwiki.apache.org/confluence/display/ARIATOSCA/Code+Style+Guidelines, 
section 5c, these should be separated into multiple lines.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-140) General purpose version comparison

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999889#comment-15999889
 ] 

ASF GitHub Bot commented on ARIA-140:
-

Github user AviaE commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/100#discussion_r115142924
  
--- Diff: aria/utils/versions.py ---
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+General-purpose version string handling
+"""
+
+import re
+
+
+_INF = float('inf')
+
+_NULL = (), _INF
--- End diff --

Why is `_NULL` not `(), 0`? Seems more natural, as it represents an invalid 
version string.


> General purpose version comparison
> --
>
> Key: ARIA-140
> URL: https://issues.apache.org/jira/browse/ARIA-140
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>Priority: Minor
>
> Add {{aria.utils.versions}} to allow for general-purpose comparison and 
> sorting of version strings.
> For now this will be used for resolving plugin specifications to plugins, but 
> it may very well have other uses in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-215) Refactor plugin-related code from ProcessExecutor into PluginManager

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999865#comment-15999865
 ] 

ASF GitHub Bot commented on ARIA-215:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/124


> Refactor plugin-related code from ProcessExecutor into PluginManager
> 
>
> Key: ARIA-215
> URL: https://issues.apache.org/jira/browse/ARIA-215
> Project: AriaTosca
>  Issue Type: Task
>Reporter: Ran Ziv
>Assignee: Ran Ziv
>Priority: Minor
>
> {{ProcessExecutor}} currently handles the code for loading a plugin into an 
> environment - this code should move into the {{PluginManager}} class instead.
> Additionally, {{PluginManager}}'s {{plugin_prefix}} method should be renamed 
> to {{plugin_dir}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-215) Refactor plugin-related code from ProcessExecutor into PluginManager

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999846#comment-15999846
 ] 

ASF GitHub Bot commented on ARIA-215:
-

GitHub user ran-z opened a pull request:

https://github.com/apache/incubator-ariatosca/pull/124

ARIA-215 Refactor plugin-related code into PluginManager

Refactored plugin-related code from ProcessExecutor into PluginManager.
Additionally, renamed plugin_prefix to plugin_dir in PluginManager.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/incubator-ariatosca 
ARIA-215-refactor-plugin-related-code

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-ariatosca/pull/124.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #124






> Refactor plugin-related code from ProcessExecutor into PluginManager
> 
>
> Key: ARIA-215
> URL: https://issues.apache.org/jira/browse/ARIA-215
> Project: AriaTosca
>  Issue Type: Task
>Reporter: Ran Ziv
>Assignee: Ran Ziv
>Priority: Minor
>
> {{ProcessExecutor}} currently handles the code for loading a plugin into an 
> environment - this code should move into the {{PluginManager}} class instead.
> Additionally, {{PluginManager}}'s {{plugin_prefix}} method should be renamed 
> to {{plugin_dir}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-160) Operation toolbelt unit tests fail spordically

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999821#comment-15999821
 ] 

ASF GitHub Bot commented on ARIA-160:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/119


> Operation toolbelt unit tests fail spordically
> --
>
> Key: ARIA-160
> URL: https://issues.apache.org/jira/browse/ARIA-160
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Maxim Orlov
>Priority: Minor
>
> The unit tests in the module {{test_toolbelt.py}} fail remote builds 
> (TravisCI, Jenkins) sporadically.
> Stack trace:
> {code}
> workflow_context = WorkflowContext(deployment_id=1, 
> workflow_name=test_workflow, execution_id=1)
> executor =  at 0x7f57f8482a50>
> def test_host_ip(workflow_context, executor):
> interface_name = 'Standard'
> operation_name = 'create'
> _, dependency_node, _, _, _ = _get_elements(workflow_context)
> inputs = {'putput': True}
> interface = mock.models.create_interface(
> dependency_node.service,
> interface_name=interface_name,
> operation_name=operation_name,
> operation_kwargs=dict(implementation=op_path(host_ip, 
> module_path=__name__),
>   inputs=inputs)
> )
> dependency_node.interfaces[interface.name] = interface
> workflow_context.model.node.update(dependency_node)
> 
> @workflow
> def basic_workflow(graph, **_):
> graph.add_tasks(
> api.task.OperationTask.for_node(
> node=dependency_node,
> interface_name=interface_name,
> operation_name=operation_name,
> inputs=inputs
> )
> )
> 
> >   execute(workflow_func=basic_workflow, 
> > workflow_context=workflow_context, executor=executor)
> tests/orchestrator/context/test_toolbelt.py:101: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tests/orchestrator/context/__init__.py:29: in execute
> eng.execute()
> aria/orchestrator/workflows/core/engine.py:59: in execute
> for task in self._ended_tasks():
> aria/orchestrator/workflows/core/engine.py:96: in 
> return (task for task in self._tasks_iter() if task.has_ended())
> aria/orchestrator/workflows/core/task.py:259: in __getattr__
> return getattr(self.model_task, attr)
> aria/orchestrator/workflows/core/task.py:178: in model_task
> return self._workflow_context.model.task.get(self._task_id)
> aria/storage/sql_mapi.py:59: in get
> result = query.first()
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first
> ret = list(self[0:1])
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in 
> __getitem__
> return list(res)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in 
> __iter__
> return self._execute_and_instances(context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in 
> _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in 
> execute
> return meth(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in 
> _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in 
> _execute_clauseelement
> compiled_sql, distilled_params
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in 
> _execute_context
> context)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in 
> _handle_dbapi_exception
> exc_info
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in 
> raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in 
> _execute_context
> context)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  0x7f57f97ee250>
> cursor = 
> statement = 'SELECT task.id AS task_id, task.name AS task_name, 
> task.implementation AS task_implementation, task.max_attempts AS t..._fk AS 
> task_execution_fk, task.relationship_fk AS task_relationship_fk 
> FROM task 
> WHERE task.id = ?
>  LIMIT ? OFFSET ?'
> parameters = (1, 1, 0)
> context =  0x7f57f83b1810>
> def do_execute(self, cursor, statement, parameters, context=None):
> >   cursor.execute(statement, parameters)
> E   InterfaceError: (sqlite3.InterfaceError) Error binding parameter 0 - 
> 

[jira] [Commented] (ARIA-156) Workflow runner continues to run when exception is raised on task_started event

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999807#comment-15999807
 ] 

ASF GitHub Bot commented on ARIA-156:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/122


> Workflow runner continues to run when exception is raised on task_started 
> event
> ---
>
> Key: ARIA-156
> URL: https://issues.apache.org/jira/browse/ARIA-156
> Project: AriaTosca
>  Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Avia Efrat
>Assignee: Avia Efrat
>
> If an exception is raised during _task_started, the engine continues to run, 
> but it finds no executable tasks or ended tasks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-156) Workflow runner continues to run when exception is raised on task_started event

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999771#comment-15999771
 ] 

ASF GitHub Bot commented on ARIA-156:
-

GitHub user AviaE opened a pull request:

https://github.com/apache/incubator-ariatosca/pull/122

ARIA-156 Better handle exceptions in the process executor

Previously, if an exception was raised during the starting of a task,
the task's process was permenantly blocked on receiving a message.

The reason was that the execption caused the 'listener thread' to
not send a response the to the task's process, as the exception was not 
handled inside the 'with' block of the listener thread.

The first change I introduced was to wrap the yielding of the message and
the response inside a try-except-finally block, so the exception will be
handled within the 'with' scope, and to ensure a response is sent to the
task's process.

The second change is to move the sending of the 'task started' message in
the task's process to a place where encountering an exception will be
handled via sending a 'task failed' message back to the listener thread.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/incubator-ariatosca 
ARIA-156-better-handle-exceptions-in-the-process-executor

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-ariatosca/pull/122.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #122


commit 7bc6d8a94a5b06d9c0719a7026077f96489e6d89
Author: Avia Efrat 
Date:   2017-05-07T08:42:58Z

ARIA-156 Better handle exceptions in the process executor

Previously, if an exception was raised during the starting of a task,
the task's process was permenantly blocked on receiving a message.

The reason was that the execption caused the 'listener thread' to
not send a response the to the task's process, as the exception was not 
handled inside the 'with' block of the listener thread.

The first change I introduced was to wrap the yielding of the message and
the response inside a try-except-finally block, so the exception will be
handled within the 'with' scope, and to ensure a response is sent to the
task's process.

The second change is to move the sending of the 'task started' message in
the task's process to a place where encountering an exception will be
handled via sending a 'task failed' message back to the listener thread.




> Workflow runner continues to run when exception is raised on task_started 
> event
> ---
>
> Key: ARIA-156
> URL: https://issues.apache.org/jira/browse/ARIA-156
> Project: AriaTosca
>  Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Avia Efrat
>Assignee: Avia Efrat
>
> If an exception is raised during _task_started, the engine continues to run, 
> but it finds no executable tasks or ended tasks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-143) Cancelling of workflow execution

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999766#comment-15999766
 ] 

ASF GitHub Bot commented on ARIA-143:
-

Github user AviaE closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/121


> Cancelling of workflow execution
> 
>
> Key: ARIA-143
> URL: https://issues.apache.org/jira/browse/ARIA-143
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Avia Efrat
>Assignee: Avia Efrat
> Fix For: 0.1.0
>
>
> Make the process of cancelling execution more robust:
> - Identify possible pitfalls and corner cases.
> - Implement force cancelling.
> **
> *Conclusions:*
> Unhandled execution status transitions resulting from cancelling an
> execution via the CLI, that we indentified and tried to address:
> TERMINATED -> CANCELLING
> You cancel the execution, but by the time we try to set the status to
> CANCELLING, the execution thread had already finished, and therefore, in
> SUCCEEDED status.
> FAILED -> CANCELLING
> You cancel the execution, but by the time we try to set the status to
> CANCELLING, the execution thread had already encountered an error, and 
> therefore, in
> FAILED state.
> TERMINATED -> CANCELLED
> Similar to #1, but with CANCELLED instead of CANCELLING.
> FAILED -> CANCELLED
> Similar to #1, but with CANCELLED instead of CANCELLING.
> In all of the above cases (#1-#4), we skip updating the execution status,
> and log that the execution already succeeded/failed before we were able
> to cancel it.
> CANCELLING -> STARTED
> You cancel the execution while it is still in pending state. Meanwhile,
> while the execution status was already set to CANCELLING, we try to set
> the execution status
> CANCELLED -> STARTED
> Similar to #5, but after the status is set to CANCELLING, it also gets
> set to CANCELLED before attempting to set it to STARTED.
> In cases #5-#6, we skip updtating the execution status, and nothing is logged.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-143) Cancelling of workflow execution

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999748#comment-15999748
 ] 

ASF GitHub Bot commented on ARIA-143:
-

GitHub user AviaE opened a pull request:

https://github.com/apache/incubator-ariatosca/pull/121

ARIA-143 Better handle exceptions in the process executor

Previously, if an exception was raised during the starting of a task,
the task's process was permenantly blocked on receiving a message.

The reason was that the execption caused the 'listener thread' to
not send a response the to the task's process, as the exception was not 
handled inside the 'with' block of the listener thread.

The first change I introduced was to wrap the yielding of the message and
the response inside a try-except-finally block, so the exception will be
handled within the 'with' scope, and to ensure a response is sent to the
task's process.

The second change is to move the sending of the 'task started' message in
the task's process to a place where encountering an exception will be
handled via sending a 'task failed' message back to the listener thread.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/incubator-ariatosca 
ARIA-143-better-handle-exceptions-in-the-process-executor

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-ariatosca/pull/121.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #121


commit e5c6522cfefffe699ab742c790d102ebce227090
Author: Avia Efrat 
Date:   2017-05-07T08:42:58Z

ARIA-143 Better handle exceptions in the process executor

Previously, if an exception was raised during the starting of a task,
the task's process was permenantly blocked on receiving a message.

The reason was that the execption caused the 'listener thread' to
not send a response the to the task's process, as the exception was not 
handled inside the 'with' block of the listener thread.

The first change I introduced was to wrap the yielding of the message and
the response inside a try-except-finally block, so the exception will be
handled within the 'with' scope, and to ensure a response is sent to the
task's process.

The second change is to move the sending of the 'task started' message in
the task's process to a place where encountering an exception will be
handled via sending a 'task failed' message back to the listener thread.




> Cancelling of workflow execution
> 
>
> Key: ARIA-143
> URL: https://issues.apache.org/jira/browse/ARIA-143
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Avia Efrat
>Assignee: Avia Efrat
> Fix For: 0.1.0
>
>
> Make the process of cancelling execution more robust:
> - Identify possible pitfalls and corner cases.
> - Implement force cancelling.
> **
> *Conclusions:*
> Unhandled execution status transitions resulting from cancelling an
> execution via the CLI, that we indentified and tried to address:
> TERMINATED -> CANCELLING
> You cancel the execution, but by the time we try to set the status to
> CANCELLING, the execution thread had already finished, and therefore, in
> SUCCEEDED status.
> FAILED -> CANCELLING
> You cancel the execution, but by the time we try to set the status to
> CANCELLING, the execution thread had already encountered an error, and 
> therefore, in
> FAILED state.
> TERMINATED -> CANCELLED
> Similar to #1, but with CANCELLED instead of CANCELLING.
> FAILED -> CANCELLED
> Similar to #1, but with CANCELLED instead of CANCELLING.
> In all of the above cases (#1-#4), we skip updating the execution status,
> and log that the execution already succeeded/failed before we were able
> to cancel it.
> CANCELLING -> STARTED
> You cancel the execution while it is still in pending state. Meanwhile,
> while the execution status was already set to CANCELLING, we try to set
> the execution status
> CANCELLED -> STARTED
> Similar to #5, but after the status is set to CANCELLING, it also gets
> set to CANCELLED before attempting to set it to STARTED.
> In cases #5-#6, we skip updtating the execution status, and nothing is logged.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-209) service-templates create-archive command help is badly formatted

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996910#comment-15996910
 ] 

ASF GitHub Bot commented on ARIA-209:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-ariatosca/pull/120


> service-templates create-archive command help is badly formatted
> 
>
> Key: ARIA-209
> URL: https://issues.apache.org/jira/browse/ARIA-209
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Ran Ziv
>Priority: Trivial
> Fix For: 0.1.0
>
>
> The {{aria service-templates create-archive}} CLI command's help is badly 
> formatted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-209) service-templates create-archive command help is badly formatted

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996778#comment-15996778
 ] 

ASF GitHub Bot commented on ARIA-209:
-

GitHub user ran-z opened a pull request:

https://github.com/apache/incubator-ariatosca/pull/120

ARIA-209 Fix  CLI command help



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/incubator-ariatosca 
ARIA-209-fix-create-archive-command-help

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-ariatosca/pull/120.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #120


commit 505df8d20be5c6ea699c8a5d0aad20a7388d6102
Author: Ran Ziv 
Date:   2017-05-04T14:07:50Z

ARIA-209 Fix  CLI command help




> service-templates create-archive command help is badly formatted
> 
>
> Key: ARIA-209
> URL: https://issues.apache.org/jira/browse/ARIA-209
> Project: AriaTosca
>  Issue Type: Bug
>Reporter: Ran Ziv
>Assignee: Ran Ziv
>Priority: Trivial
>
> The {{aria service-templates create-archive}} CLI command's help is badly 
> formatted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996648#comment-15996648
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114761560
  
--- Diff: aria/orchestrator/workflows/events_logging.py ---
@@ -36,16 +36,15 @@ def _get_task_name(task):
 @events.start_task_signal.connect
 def _start_task_handler(task, **kwargs):
 # If the task has not implementation this is an empty task.
-if task.implementation:
-task.context.logger.debug('{name} 
{task.interface_name}.{task.operation_name} has no '
-  
'implementation'.format(name=_get_task_name(task), task=task))
-else:
-task.context.logger.info('{name} 
{task.interface_name}.{task.operation_name} started...'
- .format(name=_get_task_name(task), 
task=task))
+suffix = 'started...' if task.implementation else 'has no 
implementation'
+task.context.logger.debug('{name} 
{task.interface_name}.{task.operation_name} {suffix}'
--- End diff --

task.imp -> info


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996644#comment-15996644
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114757171
  
--- Diff: aria/orchestrator/workflows/core/task.py ---
@@ -69,22 +69,22 @@ class StubTask(BaseTask):
 Base stub task for marker user tasks that only mark the start/end of a 
workflow
 or sub-workflow
 """
+STARTED = models.Task.STARTED
+SUCCESS = models.Task.SUCCESS
+
 def __init__(self, *args, **kwargs):
-super(StubTask, self).__init__(executor=base.StubExecutor(), 
*args, **kwargs)
+super(StubTask, self).__init__(executor=base.StubTaskExecutor(), 
*args, **kwargs)
 self.status = models.Task.PENDING
 self.due_at = datetime.utcnow()
 
+def execute(self):
--- End diff --

remove


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996647#comment-15996647
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114760771
  
--- Diff: aria/orchestrator/workflows/api/task.py ---
@@ -161,6 +141,27 @@ def __getattr__(self, item):
 return super(WorkflowTask, self).__getattribute__(item)
 
 
+def create_task(actor, interface_name, operation_name, **kwargs):
+"""
+This helper function enables safe creation of OperationTask, if the 
supplied interface and
--- End diff --

fix the comment


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996646#comment-15996646
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114760148
  
--- Diff: aria/orchestrator/workflows/api/task.py ---
@@ -72,63 +72,43 @@ def __init__(self,
 Do not call this constructor directly. Instead, use 
:meth:`for_node` or
 :meth:`for_relationship`.
 """
-
-actor_type = type(actor).__name__.lower()
 assert isinstance(actor, (models.Node, models.Relationship))
-assert actor_type in ('node', 'relationship')
-assert interface_name and operation_name
 super(OperationTask, self).__init__()
-
 self.actor = actor
-self.max_attempts = (self.workflow_context._task_max_attempts
- if max_attempts is None else max_attempts)
-self.retry_interval = (self.workflow_context._task_retry_interval
-   if retry_interval is None else 
retry_interval)
-self.ignore_failure = (self.workflow_context._task_ignore_failure
-   if ignore_failure is None else 
ignore_failure)
 self.interface_name = interface_name
 self.operation_name = operation_name
-self.name = OperationTask.NAME_FORMAT.format(type=actor_type,
+self.max_attempts = max_attempts or 
self.workflow_context._task_max_attempts
+self.retry_interval = retry_interval or 
self.workflow_context._task_retry_interval
+self.ignore_failure = ignore_failure or 
self.workflow_context._task_ignore_failure
--- End diff --

revert


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Add CLI display commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994953#comment-15994953
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r114553081
  
--- Diff: aria/cli/helptexts.py ---
@@ -47,3 +47,8 @@
 SORT_BY = "Key for sorting the list"
 DESCENDING = "Sort list in descending order [default: False]"
 JSON_OUTPUT = "Output logs in a consumable JSON format"
+
+DISPLAY_JSON = "Display in JSON format"
--- End diff --

lets merge this with `JSON_OUTPUT`,
the latter is a good name, and simply have it as "Format the output as JSON"


> Add CLI display commands
> 
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> {{aria service-templates display}}
> {{aria services display}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Add CLI display commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994949#comment-15994949
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r114553136
  
--- Diff: aria/cli/core/aria.py ---
@@ -325,6 +325,30 @@ def __init__(self):
 default=defaults.SERVICE_TEMPLATE_FILENAME,
 help=helptexts.SERVICE_TEMPLATE_FILENAME)
 
+self.display_json = click.option(
--- End diff --

merge this with the other JSON one


> Add CLI display commands
> 
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> {{aria service-templates display}}
> {{aria services display}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Add CLI display commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994952#comment-15994952
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r114553193
  
--- Diff: aria/cli/core/aria.py ---
@@ -325,6 +325,30 @@ def __init__(self):
 default=defaults.SERVICE_TEMPLATE_FILENAME,
 help=helptexts.SERVICE_TEMPLATE_FILENAME)
 
+self.display_json = click.option(
+'-j',
+'--json',
+is_flag=True,
+help=helptexts.DISPLAY_JSON)
+
+self.display_yaml = click.option(
--- End diff --

`yaml_output`?


> Add CLI display commands
> 
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> {{aria service-templates display}}
> {{aria services display}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Add CLI display commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994948#comment-15994948
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r114554090
  
--- Diff: aria/cli/commands/services.py ---
@@ -177,3 +179,31 @@ def inputs(service_name, model_storage, logger):
 logger.info(inputs_string.getvalue())
 else:
 logger.info('\tNo inputs')
+
+
+@services.command(name='display',
+  short_help='Display service information')
+@aria.argument('service-name')
+@aria.options.verbose()
+@aria.options.display_json
+@aria.options.display_yaml
+@aria.options.display_graph
+@aria.pass_model_storage
+@aria.pass_logger
+def display(service_name, model_storage, json, yaml, graph, logger):
+"""Display information for a specific service template
+
+`SERVICE_NAME` is the name of the service to display information on.
+"""
+logger.info('Displaying service {0}...'.format(service_name))
+service = model_storage.service.get_by_name(service_name)
+consumption.ConsumptionContext()
+if graph:
--- End diff --

General reminder that none of the methods below are standard for the CLI
Fine for now, but keep https://issues.apache.org/jira/browse/ARIA-183 in 
mind


> Add CLI display commands
> 
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> {{aria service-templates display}}
> {{aria services display}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Add CLI display commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994950#comment-15994950
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r114555739
  
--- Diff: aria/cli/commands/service_templates.py ---
@@ -181,21 +184,49 @@ def validate(service_template, 
service_template_filename,
 logger.info('Service template validated successfully')
 
 
+@service_templates.command(name='display',
+   short_help='Display service template 
information')
+@aria.argument('service-template-name')
+@aria.options.verbose()
+@aria.options.display_json
+@aria.options.display_yaml
+@aria.options.display_types
+@aria.pass_model_storage
+@aria.pass_logger
+def display(service_template_name, model_storage, json, yaml, types, 
logger):
--- End diff --

Having both `display` and `show` is confusing :/
I agree however that it doesn't make sense to make this command into a mere 
flag of `show` as they're very different.
I'm generally ok with this as is, but Maxim wanted me to suggest `aria 
parse service/service-template` as an option as well. I'll leave it for your 
consideration :)


> Add CLI display commands
> 
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> {{aria service-templates display}}
> {{aria services display}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-148) Add CLI display commands

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994951#comment-15994951
 ] 

ASF GitHub Bot commented on ARIA-148:
-

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/107#discussion_r114554248
  
--- Diff: aria/cli/commands/services.py ---
@@ -177,3 +179,31 @@ def inputs(service_name, model_storage, logger):
 logger.info(inputs_string.getvalue())
 else:
 logger.info('\tNo inputs')
+
+
+@services.command(name='display',
+  short_help='Display service information')
+@aria.argument('service-name')
+@aria.options.verbose()
+@aria.options.display_json
+@aria.options.display_yaml
+@aria.options.display_graph
+@aria.pass_model_storage
+@aria.pass_logger
+def display(service_name, model_storage, json, yaml, graph, logger):
--- End diff --

nitpicking that model_storage should probably sit after `graph`, before 
`logger`


> Add CLI display commands
> 
>
> Key: ARIA-148
> URL: https://issues.apache.org/jira/browse/ARIA-148
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Tal Liron
>Assignee: Tal Liron
>
> {{aria service-templates display}}
> {{aria services display}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994884#comment-15994884
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114545381
  
--- Diff: 
extensions/aria_extension_tosca/profiles/tosca-simple-1.0/interfaces.yaml ---
@@ -82,14 +82,14 @@ interface_types:
 relationship_edge: target
 add_target:
   description: >-
-Operation to notify the source node of a target node being added 
via a relationship.
--- End diff --

talk to tal but i think youre wrong sorry bro bruh


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994885#comment-15994885
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114545574
  
--- Diff: tests/end2end/test_hello_world.py ---
@@ -57,5 +57,5 @@ def _verify_deployed_service_in_storage(service_name, 
model_storage):
 assert service.name == service_name
 assert len(service.executions) == 1
 assert len(service.nodes) == 2
-# TODO: validate node states
--- End diff --

also remove `force_service_delete`


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994874#comment-15994874
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114544013
  
--- Diff: aria/orchestrator/workflows/events_logging.py ---
@@ -34,19 +34,25 @@ def _get_task_name(task):
 
 
 @events.start_task_signal.connect
-def _start_task_handler(task, **kwargs):
+def _start_task_handler(task, skip_logging=False, **kwargs):
+if skip_logging:
--- End diff --

remove skip_logging, add task.is_empty check


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994875#comment-15994875
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114538489
  
--- Diff: aria/orchestrator/workflows/core/translation.py ---
@@ -86,8 +88,7 @@ def _get_tasks_from_dependencies(execution_graph, 
dependencies, default=()):
 Returns task list from dependencies.
 """
 return [execution_graph.node[dependency.id
- if isinstance(dependency, 
(api.task.OperationTask,
-
api.task.StubTask))
+ if isinstance(dependency, 
api.task.OperationTask)
--- End diff --

?


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994868#comment-15994868
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114536083
  
--- Diff: aria/orchestrator/workflows/core/engine.py ---
@@ -109,12 +110,11 @@ def _tasks_iter(self):
 
self._workflow_context.model.task.refresh(task.model_task)
 yield task
 
-def _handle_executable_task(self, task):
-if isinstance(task, engine_task.StubTask):
-task.status = models.Task.SUCCESS
-else:
+@staticmethod
+def _handle_executable_task(task):
+if not isinstance(task, engine_task.StubTask):
--- End diff --

move the event raising into the executor


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994869#comment-15994869
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114535773
  
--- Diff: aria/orchestrator/workflows/api/task.py ---
@@ -87,86 +88,43 @@ def __init__(self,
if ignore_failure is None else 
ignore_failure)
 self.interface_name = interface_name
 self.operation_name = operation_name
+self.name = OperationTask.NAME_FORMAT.format(type=actor_type,
+ name=actor.name,
+ 
interface=self.interface_name,
+ 
operation=self.operation_name)
+self.is_stub = self.is_empty(self.actor, self.interface_name, 
self.operation_name)
+if self.is_stub:
+return
 
 operation = 
self.actor.interfaces[self.interface_name].operations[self.operation_name]
 self.plugin = operation.plugin
 self.inputs = modeling_utils.create_inputs(inputs or {}, 
operation.inputs)
 self.implementation = operation.implementation
-self.name = OperationTask.NAME_FORMAT.format(type=actor_type,
- name=actor.name,
- 
interface=self.interface_name,
- 
operation=self.operation_name)
 
 def __repr__(self):
 return self.name
 
-@classmethod
-def for_node(cls,
- node,
- interface_name,
- operation_name,
- max_attempts=None,
- retry_interval=None,
- ignore_failure=None,
- inputs=None):
-"""
-Creates an operation on a node.
-
-:param node: The node on which to run the operation
-:param interface_name: The interface name
-:param operation_name: The operation name within the interface
-:param max_attempts: The maximum number of attempts in case the 
operation fails
- (if not specified the defaults it taken from 
the workflow context)
-:param retry_interval: The interval in seconds between attempts 
when the operation fails
-   (if not specified the defaults it taken 
from the workflow context)
-:param ignore_failure: Whether to ignore failures
-   (if not specified the defaults it taken 
from the workflow context)
-:param inputs: Additional operation inputs
-"""
-
-assert isinstance(node, models.Node)
-return cls(
-actor=node,
-interface_name=interface_name,
-operation_name=operation_name,
-max_attempts=max_attempts,
-retry_interval=retry_interval,
-ignore_failure=ignore_failure,
-inputs=inputs)
-
-@classmethod
-def for_relationship(cls,
- relationship,
- interface_name,
- operation_name,
- max_attempts=None,
- retry_interval=None,
- ignore_failure=None,
- inputs=None):
-"""
-Creates an operation on a relationship edge.
-
-:param relationship: The relationship on which to run the operation
-:param interface_name: The interface name
-:param operation_name: The operation name within the interface
-:param max_attempts: The maximum number of attempts in case the 
operation fails
- (if not specified the defaults it taken from 
the workflow context)
-:param retry_interval: The interval in seconds between attempts 
when the operation fails
-   (if not specified the defaults it taken 
from the workflow context)
-:param ignore_failure: Whether to ignore failures
-   (if not specified the defaults it taken 
from the workflow context)
-:param inputs: Additional operation inputs
-"""
-
-assert isinstance(relationship, models.Relationship)
-return cls(
-actor=relationship,
-interface_name=interface_name,
-

[jira] [Commented] (ARIA-163) Update node state for stub tasks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARIA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994872#comment-15994872
 ] 

ASF GitHub Bot commented on ARIA-163:
-

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/117#discussion_r114537108
  
--- Diff: aria/orchestrator/workflows/core/task.py ---
@@ -130,15 +144,18 @@ def __init__(self, api_task, *args, **kwargs):
 
 task_model = create_task_model(
 name=api_task.name,
-implementation=api_task.implementation,
 actor=api_task.actor,
-inputs=api_task.inputs,
 status=base_task_model.PENDING,
 max_attempts=api_task.max_attempts,
 retry_interval=api_task.retry_interval,
 ignore_failure=api_task.ignore_failure,
-plugin=plugin,
-execution=self._workflow_context.execution
+execution=self._workflow_context.execution,
+
+# Only non-stub tasks have these fields
+plugin=getattr(api_task, 'plugin', None),
--- End diff --

update the api_task


> Update node state for stub tasks
> 
>
> Key: ARIA-163
> URL: https://issues.apache.org/jira/browse/ARIA-163
> Project: AriaTosca
>  Issue Type: Story
>Affects Versions: 0.1.0
>Reporter: Maxim Orlov
>Assignee: Maxim Orlov
>
> Currently each node that has no implementation for an operation, has its 
> operations labeled a stub tasks. Those stub tasks aren't run through the 
> normal path of the workflow engine, and thus no update to the state of the 
> node is being done. this should be fixed 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)