[jira] [Commented] (ARIA-92) Execution plugin operations default mappings
[ https://issues.apache.org/jira/browse/ARIA-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956050#comment-15956050 ] ASF GitHub Bot commented on ARIA-92: GitHub user tliron opened a pull request: https://github.com/apache/incubator-ariatosca/pull/95 ARIA-92 Automatic operation task configuration Main changes: 1. Removed `runs_on` field from task model and API 2. Two new instantiations phase, one to find the host nodes, and the second to configure operation instances for the execution plugin 3. Parser fills in `configuration` field in `OperationTemplate` using `dependencies` in TOSCA syntax 4. Fixes to reqs-and-caps bugs You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-ariatosca ARIA-92-plugin-in-implementation-string Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-ariatosca/pull/95.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #95 commit 73c1d0297812a421eee2176fcff5e5ae73980ccb Author: Tal LironDate: 2017-03-24T21:33:11Z ARIA-92 Automatic operation task configuration > Execution plugin operations default mappings > > > Key: ARIA-92 > URL: https://issues.apache.org/jira/browse/ARIA-92 > Project: AriaTosca > Issue Type: Story >Reporter: Ran Ziv >Assignee: Tal Liron > > The execution plugin serves as the default plugin, i.e. if no other plugin > was specified, it'll be used to execute scripts in operations. > These scripts will currently only execute locally. The execution plugin also > supports running scripts on remote machines (via SSH). > One option is to have the parser recognize whether the node in question is > contained inside a host node, in which case the script should be executed > remotely (by default, yet overridable by specifying the full plugin operation > mapping), and if not then it should be executed locally. > Another option is to have the user specify it using special syntax, e.g.: > "local > script.sh" and "remote > script.sh" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-ariatosca pull request #95: ARIA-92 Automatic operation task confi...
GitHub user tliron opened a pull request: https://github.com/apache/incubator-ariatosca/pull/95 ARIA-92 Automatic operation task configuration Main changes: 1. Removed `runs_on` field from task model and API 2. Two new instantiations phase, one to find the host nodes, and the second to configure operation instances for the execution plugin 3. Parser fills in `configuration` field in `OperationTemplate` using `dependencies` in TOSCA syntax 4. Fixes to reqs-and-caps bugs You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-ariatosca ARIA-92-plugin-in-implementation-string Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-ariatosca/pull/95.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #95 commit 73c1d0297812a421eee2176fcff5e5ae73980ccb Author: Tal LironDate: 2017-03-24T21:33:11Z ARIA-92 Automatic operation task configuration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
incubator-ariatosca git commit: Cleanups
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-92-plugin-in-implementation-string 669356657 -> dabfc307c Cleanups Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/dabfc307 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/dabfc307 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/dabfc307 Branch: refs/heads/ARIA-92-plugin-in-implementation-string Commit: dabfc307c0d4aa7dbbf56ca0c979b02777b71c78 Parents: 6693566 Author: Tal LironAuthored: Tue Apr 4 17:56:52 2017 -0500 Committer: Tal Liron Committed: Tue Apr 4 17:56:52 2017 -0500 -- aria/cli/dry.py | 4 +- aria/modeling/orchestration.py | 54 +- aria/modeling/service_instance.py | 80 +--- aria/modeling/service_template.py | 14 +- aria/orchestrator/execution_plugin/__init__.py | 156 --- .../execution_plugin/instantiation.py | 188 +++ .../execution_plugin/ssh/operations.py | 2 + aria/orchestrator/workflows/api/task.py | 61 ++ aria/parser/consumption/modeling.py | 12 +- .../custom_types/elasticsearch.yaml | 2 + .../multi-tier-1/custom_types/kibana.yaml | 2 + .../multi-tier-1/custom_types/logstash.yaml | 2 + .../paypalpizzastore_nodejs_app.yaml| 2 +- .../webserver-dbms-2/webserver-dbms-2.yaml | 6 +- .../simple_v1_0/modeling/capabilities.py| 5 + tests/modeling/test_models.py | 65 --- tests/orchestrator/context/test_operation.py| 3 +- tests/orchestrator/context/test_serialize.py| 3 +- tests/orchestrator/execution_plugin/test_ssh.py | 17 +- tests/orchestrator/workflows/api/test_task.py | 24 +-- .../workflows/builtin/test_execute_operation.py | 3 +- tests/orchestrator/workflows/core/test_task.py | 29 ++- .../test_task_graph_into_exececution_graph.py | 3 +- .../node-cellar/node-cellar.yaml| 2 +- 24 files changed, 413 insertions(+), 326 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/dabfc307/aria/cli/dry.py -- diff --git a/aria/cli/dry.py b/aria/cli/dry.py index 82faf42..9c38a85 100644 --- a/aria/cli/dry.py +++ b/aria/cli/dry.py @@ -49,8 +49,8 @@ def convert_operation_to_dry(oper): Converts a single :class:`Operation` to run dryly. """ -plugin = oper.plugin_specification.name \ -if oper.plugin_specification is not None else None +plugin = oper.plugin.name \ +if oper.plugin is not None else None if oper.inputs is None: oper.inputs = OrderedDict() oper.inputs['_implementation'] = models.Parameter(name='_implementation', http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/dabfc307/aria/modeling/orchestration.py -- diff --git a/aria/modeling/orchestration.py b/aria/modeling/orchestration.py index b5c735d..d29f9f8 100644 --- a/aria/modeling/orchestration.py +++ b/aria/modeling/orchestration.py @@ -200,7 +200,44 @@ class PluginBase(ModelMixin): class TaskBase(ModelMixin): """ -A Model which represents an task +Represents the smallest unit of stateful execution in ARIA. The task state includes inputs, +outputs, as well as an atomic status, ensuring that the task can only be running once at any +given time. + +Tasks may be "one shot" or may be configured to run repeatedly in the case of failure. + +Tasks are often based on :class:`Operation`, and thus act on either a :class:`Node` or a +:class:`Relationship`, however this is not required. + +:ivar node: The node actor (optional) +:vartype node: :class:`Node` +:ivar relationship: The relationship actor (optional) +:vartype relationship: :class:`Relationship` +:ivar plugin: The implementing plugin (set to None for default execution plugin) +:vartype plugin: :class:`Plugin` +:ivar inputs: Parameters that can be used by this task +:vartype inputs: {basestring: :class:`Parameter`} +:ivar implementation: Python path to an ``@operation`` function +:vartype implementation: basestring +:ivar max_attempts: Maximum number of retries allowed in case of failure +:vartype max_attempts: int +:ivar retry_interval: Interval between retries (in seconds) +:vartype retry_interval: int +:ivar ignore_failure: Set to True to ignore failures +:vartype ignore_failure: bool +:ivar due_at: Timestamp to start the task +:vartype due_at: datetime +:ivar
incubator-ariatosca git commit: used right logger in dry executor
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-48-aria-cli 8b4cd4ac9 -> 0d46af2d0 used right logger in dry executor Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/0d46af2d Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/0d46af2d Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/0d46af2d Branch: refs/heads/ARIA-48-aria-cli Commit: 0d46af2d04d37efcf51bafe1d352772a84af4120 Parents: 8b4cd4a Author: Ran ZivAuthored: Tue Apr 4 14:26:24 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 14:26:24 2017 +0300 -- aria/orchestrator/workflows/executor/dry.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/0d46af2d/aria/orchestrator/workflows/executor/dry.py -- diff --git a/aria/orchestrator/workflows/executor/dry.py b/aria/orchestrator/workflows/executor/dry.py index 69ce53c..c2b5341 100644 --- a/aria/orchestrator/workflows/executor/dry.py +++ b/aria/orchestrator/workflows/executor/dry.py @@ -40,7 +40,7 @@ class DryExecutor(BaseExecutor): implementation += task.implementation inputs = Parameter.unwrap_dict(task.inputs) -self.logger.info( +task.context.logger.info( 'Executing {actor_type} {actor_name} operation {interface_name} {operation_name}: ' '{implementation} (Inputs: {inputs})' .format(actor_type=actor_type, actor_name=task.actor.name,
[07/24] incubator-ariatosca git commit: ARIA-48 cli
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d8722826/aria/orchestrator/context/common.py -- diff --git a/aria/orchestrator/context/common.py b/aria/orchestrator/context/common.py index 127641f..f631e79 100644 --- a/aria/orchestrator/context/common.py +++ b/aria/orchestrator/context/common.py @@ -68,17 +68,6 @@ class BaseContext(object): self._workdir = workdir self.logger = None -def _create_execution(self): -now = datetime.utcnow() -execution = self.model.execution.model_cls( -service_instance=self.service_instance, -workflow_name=self._workflow_name, -created_at=now, -parameters=self.parameters, -) -self.model.execution.put(execution) -return execution.id - def _register_logger(self, logger_name=None, level=None, task_id=None): self.logger = self.PrefixedLogger(logging.getLogger(logger_name or self.__class__.__name__), self.logging_id, @@ -168,13 +157,13 @@ class BaseContext(object): Download a blueprint resource from the resource storage """ try: -self.resource.deployment.download(entry_id=str(self.service.id), - destination=destination, - path=path) +self.resource.service.download(entry_id=str(self.service.id), + destination=destination, + path=path) except exceptions.StorageError: - self.resource.blueprint.download(entry_id=str(self.service_template.id), - destination=destination, - path=path) + self.resource.service_template.download(entry_id=str(self.service_template.id), +destination=destination, +path=path) def download_resource_and_render(self, destination, path=None, variables=None): """ @@ -193,9 +182,9 @@ class BaseContext(object): Read a deployment resource as string from the resource storage """ try: -return self.resource.deployment.read(entry_id=str(self.service.id), path=path) +return self.resource.service.read(entry_id=str(self.service.id), path=path) except exceptions.StorageError: -return self.resource.deployment.read(entry_id=str(self.service_template.id), path=path) +return self.resource.service_template.read(entry_id=str(self.service_template.id), path=path) def get_resource_and_render(self, path=None, variables=None): """ http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d8722826/aria/orchestrator/context/workflow.py -- diff --git a/aria/orchestrator/context/workflow.py b/aria/orchestrator/context/workflow.py index 5f86d9d..bc9f653 100644 --- a/aria/orchestrator/context/workflow.py +++ b/aria/orchestrator/context/workflow.py @@ -31,11 +31,11 @@ class WorkflowContext(BaseContext): """ def __init__(self, workflow_name, + execution_id, parameters=None, task_max_attempts=1, task_retry_interval=0, task_ignore_failure=False, - execution_id=None, *args, **kwargs): super(WorkflowContext, self).__init__(*args, **kwargs) self._workflow_name = workflow_name @@ -43,28 +43,15 @@ class WorkflowContext(BaseContext): self._task_max_attempts = task_max_attempts self._task_retry_interval = task_retry_interval self._task_ignore_failure = task_ignore_failure -# TODO: execution creation should happen somewhere else -# should be moved there, when such logical place exists -self._execution_id = execution_id or self._create_execution() +self._execution_id = execution_id self._register_logger() def __repr__(self): return ( '{name}(deployment_id={self._service_id}, ' -'workflow_name={self._workflow_name}'.format( +'workflow_name={self._workflow_name}, execution_id={self._execution_id})'.format( name=self.__class__.__name__, self=self)) -def _create_execution(self): -now = datetime.utcnow() -execution = self.model.execution.model_cls( -service=self.service, -workflow_name=self._workflow_name, -created_at=now, -parameters=self.parameters, -) -self.model.execution.put(execution) -return execution.id - @property
[09/24] incubator-ariatosca git commit: ARIA-48 cli
ARIA-48 cli Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/d8722826 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/d8722826 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/d8722826 Branch: refs/heads/ARIA-48-aria-cli Commit: d872282688a50178d396fb3cbd8946b0ec37d80a Parents: e7ffc73 Author: Ran ZivAuthored: Tue Mar 28 12:17:46 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/__init__.py| 2 +- aria/cli/VERSION| 3 + aria/cli/args_parser.py | 269 - aria/cli/cli.py | 113 aria/cli/cli/__init__.py| 14 + aria/cli/cli/aria.py| 453 +++ aria/cli/cli/helptexts.py | 56 ++ aria/cli/colorful_event.py | 152 ++ aria/cli/commands.py| 546 --- aria/cli/commands/__init__.py | 14 + aria/cli/commands/executions.py | 172 ++ aria/cli/commands/logs.py | 68 +++ aria/cli/commands/node_templates.py | 104 aria/cli/commands/nodes.py | 96 aria/cli/commands/plugins.py| 145 + aria/cli/commands/service_templates.py | 207 +++ aria/cli/commands/services.py | 175 ++ aria/cli/commands/workflows.py | 107 aria/cli/config.py | 46 -- aria/cli/config/__init__.py | 14 + aria/cli/config/config.py | 70 +++ aria/cli/config/config_template.yaml| 12 + aria/cli/constants.py | 18 + aria/cli/csar.py| 11 +- aria/cli/dry.py | 88 --- aria/cli/env.py | 118 aria/cli/exceptions.py | 54 +- aria/cli/inputs.py | 118 aria/cli/logger.py | 113 aria/cli/main.py| 73 +++ aria/cli/service_template_utils.py | 140 + aria/cli/storage.py | 95 aria/cli/table.py | 90 +++ aria/cli/utils.py | 152 ++ aria/core.py| 116 aria/exceptions.py | 16 + aria/modeling/__init__.py | 2 + aria/modeling/exceptions.py | 18 + aria/modeling/orchestration.py | 14 +- aria/modeling/service_changes.py| 10 +- aria/modeling/service_instance.py | 15 +- aria/modeling/service_template.py | 16 +- aria/modeling/utils.py | 87 ++- aria/orchestrator/context/common.py | 27 +- aria/orchestrator/context/workflow.py | 19 +- .../execution_plugin/ctx_proxy/server.py| 3 +- aria/orchestrator/runner.py | 101 aria/orchestrator/workflow_runner.py| 147 + aria/orchestrator/workflows/api/task.py | 41 +- aria/orchestrator/workflows/builtin/__init__.py | 1 + .../workflows/builtin/execute_operation.py | 15 +- aria/orchestrator/workflows/builtin/utils.py| 86 ++- aria/storage/core.py| 5 + aria/utils/archive.py | 63 +++ aria/utils/exceptions.py| 11 + aria/utils/file.py | 13 + aria/utils/formatting.py| 43 ++ aria/utils/threading.py | 24 + aria/utils/type.py | 57 ++ .../simple_v1_0/modeling/__init__.py| 3 +- requirements.in | 5 + setup.py| 2 +- tests/mock/context.py | 7 +- tests/mock/models.py| 13 +- tests/modeling/test_models.py | 10 +- tests/orchestrator/context/test_operation.py| 45 +- .../context/test_resource_render.py | 4 +- tests/orchestrator/context/test_serialize.py| 4 +- tests/orchestrator/context/test_toolbelt.py | 11 +- tests/orchestrator/context/test_workflow.py | 10 +- .../orchestrator/execution_plugin/test_local.py | 10 +- tests/orchestrator/execution_plugin/test_ssh.py | 2 +- .../workflows/builtin/test_execute_operation.py | 3 +-
[08/24] incubator-ariatosca git commit: ARIA-48 cli
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d8722826/aria/cli/commands/service_templates.py -- diff --git a/aria/cli/commands/service_templates.py b/aria/cli/commands/service_templates.py new file mode 100644 index 000..b855529 --- /dev/null +++ b/aria/cli/commands/service_templates.py @@ -0,0 +1,207 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +import os +import json + +from .. import utils +from .. import csar +from ..cli import aria +from .. import service_template_utils +from ..table import print_data +from ..exceptions import AriaCliError +from ...core import Core +from ...exceptions import AriaException + + +DESCRIPTION_LIMIT = 20 +SERVICE_TEMPLATE_COLUMNS = \ +['id', 'name', 'main_file_name', 'created_at', 'updated_at'] + + +@aria.group(name='service-templates') +@aria.options.verbose() +def service_templates(): +"""Handle service templates on the manager +""" +pass + + +@service_templates.command(name='show', + short_help='Show service template information') +@aria.argument('service-template-id') +@aria.options.verbose() +@aria.pass_model_storage +@aria.pass_logger +def show(service_template_id, model_storage, logger): +"""Show information for a specific service templates + +`SERVICE_TEMPLATE_ID` is the id of the service template to show information on. +""" +logger.info('Showing service template {0}...'.format(service_template_id)) +service_template = model_storage.service_template.get(service_template_id) +services = [d.to_dict() for d in service_template.services] +service_template_dict = service_template.to_dict() +service_template_dict['#services'] = len(services) +columns = SERVICE_TEMPLATE_COLUMNS + ['#services'] +print_data(columns, service_template_dict, 'Service-template:', max_width=50) + +logger.info('Description:') + logger.info('{0}\n'.format(service_template_dict['description'].encode('UTF-8') or '')) + +logger.info('Existing services:') +logger.info('{0}\n'.format(json.dumps([d['name'] for d in services]))) + + +@service_templates.command(name='list', + short_help='List service templates') +@aria.options.sort_by() +@aria.options.descending +@aria.options.verbose() +@aria.pass_model_storage +@aria.pass_logger +def list(sort_by, descending, model_storage, logger): +"""List all service templates +""" +def trim_description(service_template): +if service_template['description'] is not None: +if len(service_template['description']) >= DESCRIPTION_LIMIT: +service_template['description'] = '{0}..'.format( +service_template['description'][:DESCRIPTION_LIMIT - 2]) +else: +service_template['description'] = '' +return service_template + +logger.info('Listing all service templates...') +service_templates = [trim_description(b.to_dict()) for b in model_storage.service_template.list( +sort=utils.storage_sort_param(sort_by, descending))] +print_data(SERVICE_TEMPLATE_COLUMNS, service_templates, 'Service templates:') + + +@service_templates.command(name='store', + short_help='Store a service template') +@aria.argument('service-template-path') +@aria.argument('service-template-name') +@aria.options.verbose() +@aria.pass_model_storage +@aria.pass_resource_storage +@aria.pass_plugin_manager +@aria.pass_logger +def store(service_template_path, service_template_name, model_storage, resource_storage, + plugin_manager, logger): +"""Store a service template + +`SERVICE_TEMPLATE_PATH` is the path of the service template to store. + +`SERVICE_TEMPLATE_NAME` is the name of the service template to store. +""" +logger.info('Storing service template {0}...'.format(service_template_name)) + +service_template_path = service_template_utils.get(service_template_path) +core = Core(model_storage, resource_storage, plugin_manager) +core.create_service_template(service_template_path, + os.path.dirname(service_template_path), +
[20/24] incubator-ariatosca git commit: Fix issue where a service name was passed instead of a service id
Fix issue where a service name was passed instead of a service id Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/a26f245e Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/a26f245e Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/a26f245e Branch: refs/heads/ARIA-48-aria-cli Commit: a26f245e83ac9e00d6ea5d9dd1248cf461bd3888 Parents: 375228b Author: Avia EfratAuthored: Mon Apr 3 17:09:25 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/services.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/a26f245e/aria/cli/commands/services.py -- diff --git a/aria/cli/commands/services.py b/aria/cli/commands/services.py index e73c43b..9f9d104 100644 --- a/aria/cli/commands/services.py +++ b/aria/cli/commands/services.py @@ -97,7 +97,8 @@ def create(service_template_name, try: core = Core(model_storage, resource_storage, plugin_manager) -service = core.create_service(service_template_name, inputs, service_name) +service_template = model_storage.service_template.get_by_name(service_template_name) +service = core.create_service(service_template.id, inputs, service_name) except storage_exceptions.StorageError as e: handle_storage_exception(e, 'service', service_name) except AriaException as e:
[05/24] incubator-ariatosca git commit: Only print description in `service-templates` show if one exists
Only print description in `service-templates` show if one exists Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/298885fb Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/298885fb Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/298885fb Branch: refs/heads/ARIA-48-aria-cli Commit: 298885fbfdb6abdf68f643b579b0aca90c75dfd7 Parents: 2ffefc4 Author: Avia EfratAuthored: Sun Apr 2 18:17:31 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/service_templates.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/298885fb/aria/cli/commands/service_templates.py -- diff --git a/aria/cli/commands/service_templates.py b/aria/cli/commands/service_templates.py index 823a6e8..af6252a 100644 --- a/aria/cli/commands/service_templates.py +++ b/aria/cli/commands/service_templates.py @@ -61,8 +61,9 @@ def show(service_template_id, model_storage, logger): columns = SERVICE_TEMPLATE_COLUMNS + ['#services'] print_data(columns, service_template_dict, 'Service-template:', max_width=50) -logger.info('Description:') - logger.info('{0}\n'.format(service_template_dict['description'].encode('UTF-8') or '')) +if service_template_dict['description'] is not None: +logger.info('Description:') + logger.info('{0}\n'.format(service_template_dict['description'].encode('UTF-8') or '')) logger.info('Existing services:') logger.info('{0}\n'.format(json.dumps([d['name'] for d in services])))
[15/24] incubator-ariatosca git commit: fixed plugins command in CLI
fixed plugins command in CLI Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/bf0adb54 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/bf0adb54 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/bf0adb54 Branch: refs/heads/ARIA-48-aria-cli Commit: bf0adb540618d264ffa55267792b7818c6cedc92 Parents: 2214a98 Author: Ran ZivAuthored: Sun Apr 2 22:40:15 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/plugins.py | 80 +-- aria/cli/env.py | 2 +- aria/orchestrator/plugin.py | 3 +- 3 files changed, 36 insertions(+), 49 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bf0adb54/aria/cli/commands/plugins.py -- diff --git a/aria/cli/commands/plugins.py b/aria/cli/commands/plugins.py index d31aa99..4d568d7 100644 --- a/aria/cli/commands/plugins.py +++ b/aria/cli/commands/plugins.py @@ -13,18 +13,16 @@ # See the License for the specific language governing permissions and # limitations under the License. -import tarfile +import zipfile from ..table import print_data -from ..cli import helptexts, aria +from ..cli import aria from ..exceptions import AriaCliError from ..utils import storage_sort_param -PLUGIN_COLUMNS = ['id', 'package_name', 'package_version', 'distribution', - 'supported_platform', 'distribution_release', 'uploaded_at'] -EXCLUDED_COLUMNS = ['archive_name', 'distribution_version', 'excluded_wheels', -'package_source', 'supported_py_versions', 'wheels'] +PLUGIN_COLUMNS = ['id', 'package_name', 'package_version', 'supported_platform', + 'distribution', 'distribution_release', 'uploaded_at'] @aria.group(name='plugins') @@ -45,24 +43,23 @@ def validate(plugin_path, logger): This will try to validate the plugin's archive is not corrupted. A valid plugin is a wagon (http://github.com/cloudify-cosomo/wagon) -in the tar.gz format (suffix may also be .wgn). +in the zip format (suffix may also be .wgn). `PLUGIN_PATH` is the path to wagon archive to validate. """ logger.info('Validating plugin {0}...'.format(plugin_path)) -if not tarfile.is_tarfile(plugin_path): +if not zipfile.is_zipfile(plugin_path): raise AriaCliError( 'Archive {0} is of an unsupported type. Only ' -'tar.gz/wgn is allowed'.format(plugin_path)) -with tarfile.open(plugin_path) as tar: -tar_members = tar.getmembers() -package_json_path = "{0}/{1}".format( -tar_members[0].name, 'package.json') -# TODO: Find a better way to validate a plugin. +'zip/wgn is allowed'.format(plugin_path)) +with zipfile.ZipFile(plugin_path, 'r') as zip_file: +infos = zip_file.infolist() try: -tar.getmember(package_json_path) -except KeyError: +package_name = infos[0].filename[:infos[0].filename.index('/')] +package_json_path = "{0}/{1}".format(package_name, 'package.json') +zip_file.getinfo(package_json_path) +except (KeyError, ValueError, IndexError): raise AriaCliError( 'Failed to validate plugin {0} ' '(package.json was not found in archive)'.format(plugin_path)) @@ -70,20 +67,20 @@ def validate(plugin_path, logger): logger.info('Plugin validated successfully') -@plugins.command(name='delete', - short_help='Delete a plugin') -@aria.argument('plugin-id') -@aria.options.verbose() -@aria.pass_model_storage -@aria.pass_logger -def delete(plugin_id, model_storage, logger): -"""Delete a plugin - -`PLUGIN_ID` is the id of the plugin to delete. -""" -logger.info('Deleting plugin {0}...'.format(plugin_id)) -model_storage.plugin.delete(plugin_id=plugin_id) -logger.info('Plugin deleted') +# @plugins.command(name='delete', +# short_help='Delete a plugin') +# @aria.argument('plugin-id') +# @aria.options.verbose() +# @aria.pass_model_storage +# @aria.pass_logger +# def delete(plugin_id, model_storage, logger): +# """Delete a plugin +# +# `PLUGIN_ID` is the id of the plugin to delete. +# """ +# logger.info('Deleting plugin {0}...'.format(plugin_id)) +# model_storage.plugin.delete(plugin_id=plugin_id) +# logger.info('Plugin deleted') @plugins.command(name='install', @@ -91,13 +88,14 @@ def delete(plugin_id, model_storage, logger): @aria.argument('plugin-path') @aria.options.verbose() @aria.pass_context +@aria.pass_plugin_manager
[14/24] incubator-ariatosca git commit: small fixes after rebase from master
small fixes after rebase from master Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/2ffefc49 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/2ffefc49 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/2ffefc49 Branch: refs/heads/ARIA-48-aria-cli Commit: 2ffefc498812a8c3f5c02e994d0333296ec2e180 Parents: 060f8cc Author: Ran ZivAuthored: Sun Apr 2 17:53:13 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/node_templates.py | 2 +- aria/core.py| 8 +++- 2 files changed, 4 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ffefc49/aria/cli/commands/node_templates.py -- diff --git a/aria/cli/commands/node_templates.py b/aria/cli/commands/node_templates.py index 5614aee..7365f0c 100644 --- a/aria/cli/commands/node_templates.py +++ b/aria/cli/commands/node_templates.py @@ -62,7 +62,7 @@ def show(node_template_id, model_storage, logger): logger.info('\tNo properties') # print node IDs -nodes = node_template.nodes.all() +nodes = node_template.nodes logger.info('Nodes:') if nodes: for node in nodes: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/2ffefc49/aria/core.py -- diff --git a/aria/core.py b/aria/core.py index 96a967f..f2dfb09 100644 --- a/aria/core.py +++ b/aria/core.py @@ -61,7 +61,7 @@ class Core(object): def delete_service_template(self, service_template_id): service_template = self.model_storage.service_template.get(service_template_id) -if service_template.services.all(): +if service_template.services: raise exceptions.DependentServicesError( "Can't delete service template {0} - Service template has existing services") @@ -89,16 +89,14 @@ class Core(object): def delete_service(self, service_name, force=False): service = self.model_storage.service.get_by_name(service_name) -active_executions = [e for e in service.executions - if e.status not in models.Execution.ACTIVE_STATES] +active_executions = [e for e in service.executions if e.is_active()] if active_executions: raise exceptions.DependentActiveExecutionsError( "Can't delete service {0} - there is an active execution for this service. " "Active execution id: {1}".format(service_name, active_executions[0].id)) if not force: -available_nodes = [n for n in service.nodes.values() - if n.state not in ('deleted', 'errored')] +available_nodes = [n for n in service.nodes.values() if n.is_available()] if available_nodes: raise exceptions.DependentAvailableNodesError( "Can't delete service {0} - there are available nodes for this service. "
[03/24] incubator-ariatosca git commit: created dry-run executor
created dry-run executor Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/c5624260 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/c5624260 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/c5624260 Branch: refs/heads/ARIA-48-aria-cli Commit: c56242604f4025e7e62d8304dba89bf2e80ff1e0 Parents: 108d8a6 Author: Ran ZivAuthored: Tue Apr 4 12:02:11 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/cli/aria.py| 5 +++ aria/cli/cli/helptexts.py | 2 + aria/cli/commands/executions.py | 15 +-- aria/orchestrator/workflow_runner.py| 5 ++- aria/orchestrator/workflows/executor/dry.py | 54 5 files changed, 76 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c5624260/aria/cli/cli/aria.py -- diff --git a/aria/cli/cli/aria.py b/aria/cli/cli/aria.py index baa72eb..1664ce5 100644 --- a/aria/cli/cli/aria.py +++ b/aria/cli/cli/aria.py @@ -304,6 +304,11 @@ class Options(object): is_flag=True, help=helptexts.JSON_OUTPUT) +self.dry_execution = click.option( +'--dry', +is_flag=True, +help=helptexts.DRY_EXECUTION) + self.init_hard_reset = click.option( '--hard', is_flag=True, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c5624260/aria/cli/cli/helptexts.py -- diff --git a/aria/cli/cli/helptexts.py b/aria/cli/cli/helptexts.py index 02519cb..0d66d6b 100644 --- a/aria/cli/cli/helptexts.py +++ b/aria/cli/cli/helptexts.py @@ -32,6 +32,8 @@ HARD_RESET = "Hard reset the configuration, including coloring and loggers" ENABLE_COLORS = "Enable colors in logger (use --hard when working with" \ " an initialized environment) [default: False]" +DRY_EXECUTION = "Execute a workflow dry run (prints operations information without causing side " \ +"effects)" SERVICE_TEMPLATE_FILENAME = ( "The name of the archive's main service template file. " "This is only relevant if uploading an archive") http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c5624260/aria/cli/commands/executions.py -- diff --git a/aria/cli/commands/executions.py b/aria/cli/commands/executions.py index 6d8b949..82ee51a 100644 --- a/aria/cli/commands/executions.py +++ b/aria/cli/commands/executions.py @@ -18,6 +18,7 @@ from ..table import print_data from ..cli import aria from ...modeling.models import Execution from ...orchestrator.workflow_runner import WorkflowRunner +from ...orchestrator.workflows.executor.dry import DryExecutor from ...utils import formatting from ...utils import threading @@ -101,6 +102,7 @@ def list(service_name, @aria.argument('workflow-name') @aria.options.service_name(required=True) @aria.options.inputs +@aria.options.dry_execution @aria.options.task_max_attempts() @aria.options.task_retry_interval() @aria.options.verbose() @@ -111,6 +113,7 @@ def list(service_name, def start(workflow_name, service_name, inputs, + dry, task_max_attempts, task_retry_interval, model_storage, @@ -119,19 +122,21 @@ def start(workflow_name, logger): """Execute a workflow -`WORKFLOW_ID` is the id of the workflow to execute (e.g. `uninstall`) +`WORKFLOW_NAME` is the name of the workflow to execute (e.g. `uninstall`) """ +executor = DryExecutor() if dry else None # use WorkflowRunner's default executor + workflow_runner = \ WorkflowRunner(workflow_name, service_name, inputs, model_storage, resource_storage, plugin_manager, - task_max_attempts, task_retry_interval) + executor, task_max_attempts, task_retry_interval) execution_thread_name = '{0}_{1}'.format(service_name, workflow_name) execution_thread = threading.ExceptionThread(target=workflow_runner.execute, name=execution_thread_name) execution_thread.daemon = True # allows force-cancel to exit immediately -logger.info('Starting execution. Press Ctrl+C cancel') +logger.info('Starting {0}execution. Press Ctrl+C cancel'.format('dry ' if dry else '')) execution_thread.start() try: while execution_thread.is_alive(): @@ -148,6 +153,10 @@ def start(workflow_name,
[10/24] incubator-ariatosca git commit: Make Execution, Task and Node status properties into methods
Make Execution, Task and Node status properties into methods Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/c4ca1e8f Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/c4ca1e8f Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/c4ca1e8f Branch: refs/heads/ARIA-48-aria-cli Commit: c4ca1e8f98c1b3ac0c974da06f238be989c394a6 Parents: d872282 Author: Avia EfratAuthored: Sun Apr 2 16:11:43 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/modeling/orchestration.py | 6 +- aria/modeling/service_instance.py | 1 - aria/orchestrator/workflows/core/engine.py | 6 +++--- aria/orchestrator/workflows/core/task.py | 2 -- 4 files changed, 4 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c4ca1e8f/aria/modeling/orchestration.py -- diff --git a/aria/modeling/orchestration.py b/aria/modeling/orchestration.py index 15abde4..3ad6b58 100644 --- a/aria/modeling/orchestration.py +++ b/aria/modeling/orchestration.py @@ -98,13 +98,11 @@ class ExecutionBase(ModelMixin): status = Column(Enum(*STATES, name='execution_status'), default=PENDING) workflow_name = Column(Text) -@property def has_ended(self): return self.status in self.END_STATES -@property def is_active(self): -return not self.has_ended +return not self.has_ended() @declared_attr def logs(cls): @@ -290,11 +288,9 @@ class TaskBase(ModelMixin): implementation = Column(String) _runs_on = Column(Enum(*RUNS_ON, name='runs_on'), name='runs_on') -@property def has_ended(self): return self.status in [self.SUCCESS, self.FAILED] -@property def is_waiting(self): return self.status in [self.PENDING, self.RETRYING] http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c4ca1e8f/aria/modeling/service_instance.py -- diff --git a/aria/modeling/service_instance.py b/aria/modeling/service_instance.py index 48615af..a8fe71b 100644 --- a/aria/modeling/service_instance.py +++ b/aria/modeling/service_instance.py @@ -415,7 +415,6 @@ class NodeBase(InstanceModelMixin): except KeyError: return None -@property def is_available(self): return self.state not in [self.INITIAL, self.DELETED, self.ERROR] http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c4ca1e8f/aria/orchestrator/workflows/core/engine.py -- diff --git a/aria/orchestrator/workflows/core/engine.py b/aria/orchestrator/workflows/core/engine.py index d32abb8..0503142 100644 --- a/aria/orchestrator/workflows/core/engine.py +++ b/aria/orchestrator/workflows/core/engine.py @@ -88,12 +88,12 @@ class Engine(logger.LoggerMixin): def _executable_tasks(self): now = datetime.utcnow() return (task for task in self._tasks_iter() -if task.is_waiting and +if task.is_waiting() and task.due_at <= now and not self._task_has_dependencies(task)) def _ended_tasks(self): -return (task for task in self._tasks_iter() if task.has_ended) +return (task for task in self._tasks_iter() if task.has_ended()) def _task_has_dependencies(self, task): return len(self._execution_graph.pred.get(task.id, {})) > 0 @@ -105,7 +105,7 @@ class Engine(logger.LoggerMixin): for _, data in self._execution_graph.nodes_iter(data=True): task = data['task'] if isinstance(task, engine_task.OperationTask): -if not task.model_task.has_ended: +if not task.model_task.has_ended(): self._workflow_context.model.task.refresh(task.model_task) yield task http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c4ca1e8f/aria/orchestrator/workflows/core/task.py -- diff --git a/aria/orchestrator/workflows/core/task.py b/aria/orchestrator/workflows/core/task.py index aa8963f..540988c 100644 --- a/aria/orchestrator/workflows/core/task.py +++ b/aria/orchestrator/workflows/core/task.py @@ -69,11 +69,9 @@ class StubTask(BaseTask): self.status = models.Task.PENDING self.due_at = datetime.utcnow() -@property def has_ended(self): return self.status in [models.Task.SUCCESS, models.Task.FAILED] -@property def is_waiting(self): return
[17/24] incubator-ariatosca git commit: moved service inputs population earlier in the process
moved service inputs population earlier in the process Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/349d4d05 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/349d4d05 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/349d4d05 Branch: refs/heads/ARIA-48-aria-cli Commit: 349d4d05c3faf725da95e00404f346272d388393 Parents: 72c9969 Author: Ran ZivAuthored: Mon Apr 3 01:01:00 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/core.py | 6 +- aria/modeling/service_template.py | 8 +--- 2 files changed, 6 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/349d4d05/aria/core.py -- diff --git a/aria/core.py b/aria/core.py index 5368f9c..6be6fd1 100644 --- a/aria/core.py +++ b/aria/core.py @@ -74,11 +74,7 @@ class Core(object): # creating an empty ConsumptionContext, initiating a threadlocal context ConsumptionContext() with self.model_storage._all_api_kwargs['session'].no_autoflush: -service = service_template.instantiate(None) - -template_inputs = service_template.inputs -service.inputs = modeling_utils.create_inputs(inputs, template_inputs) -# TODO: now that we have inputs, we should scan properties and inputs and evaluate functions +service = service_template.instantiate(None, inputs) service.name = service_name or '{0}_{1}'.format(service_template_name, service.id) self.model_storage.service.put(service) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/349d4d05/aria/modeling/service_template.py -- diff --git a/aria/modeling/service_template.py b/aria/modeling/service_template.py index 86cf81a..1c6d393 100644 --- a/aria/modeling/service_template.py +++ b/aria/modeling/service_template.py @@ -279,7 +279,7 @@ class ServiceTemplateBase(TemplateModelMixin): ('interface_types', formatting.as_raw(self.interface_types)), ('artifact_types', formatting.as_raw(self.artifact_types -def instantiate(self, container): +def instantiate(self, container, inputs=None): from . import models context = ConsumptionContext.get_thread_local() now = datetime.now() @@ -287,9 +287,12 @@ class ServiceTemplateBase(TemplateModelMixin): updated_at=now, description=deepcopy_with_locators(self.description), service_template=self) - context.modeling.instance = service +# utils.instantiate_dict(self, service.inputs, self.inputs) +service.inputs = utils.create_inputs(inputs or {}, self.inputs) +# TODO: now that we have inputs, we should scan properties and inputs and evaluate functions + utils.instantiate_dict(self, service.meta_data, self.meta_data) for node_template in self.node_templates.itervalues(): @@ -305,7 +308,6 @@ class ServiceTemplateBase(TemplateModelMixin): if self.substitution_template is not None: service.substitution = self.substitution_template.instantiate(container) -utils.instantiate_dict(self, service.inputs, self.inputs) utils.instantiate_dict(self, service.outputs, self.outputs) return service
[21/24] incubator-ariatosca git commit: core.py now works with ids only again; removed unnecessary NotFoundError exception clauses from cli
core.py now works with ids only again; removed unnecessary NotFoundError exception clauses from cli Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/38d58d4d Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/38d58d4d Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/38d58d4d Branch: refs/heads/ARIA-48-aria-cli Commit: 38d58d4d28cd7be21c152de767ccc567622fb263 Parents: 141dbb0 Author: Ran ZivAuthored: Mon Apr 3 15:10:51 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/executions.py| 16 aria/cli/commands/logs.py | 16 +++- aria/cli/commands/node_templates.py| 16 aria/cli/commands/nodes.py | 14 +++--- aria/cli/commands/service_templates.py | 9 ++--- aria/cli/commands/services.py | 9 ++--- aria/cli/commands/workflows.py | 21 - aria/core.py | 16 +++- 8 files changed, 37 insertions(+), 80 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/38d58d4d/aria/cli/commands/executions.py -- diff --git a/aria/cli/commands/executions.py b/aria/cli/commands/executions.py index fd47eb3..6d8b949 100644 --- a/aria/cli/commands/executions.py +++ b/aria/cli/commands/executions.py @@ -16,9 +16,7 @@ from .. import utils from ..table import print_data from ..cli import aria -from ..exceptions import AriaCliError from ...modeling.models import Execution -from ...storage import exceptions as storage_exceptions from ...orchestrator.workflow_runner import WorkflowRunner from ...utils import formatting from ...utils import threading @@ -46,11 +44,8 @@ def show(execution_id, model_storage, logger): `EXECUTION_ID` is the execution to get information on. """ -try: -logger.info('Showing execution {0}'.format(execution_id)) -execution = model_storage.execution.get(execution_id) -except storage_exceptions.NotFoundError: -raise AriaCliError('Execution {0} not found'.format(execution_id)) +logger.info('Showing execution {0}'.format(execution_id)) +execution = model_storage.execution.get(execution_id) print_data(EXECUTION_COLUMNS, execution.to_dict(), 'Execution:', max_width=50) @@ -88,11 +83,8 @@ def list(service_name, if service_name: logger.info('Listing executions for service {0}...'.format( service_name)) -try: -service = model_storage.service.get_by_name(service_name) -filters = dict(service=service) -except storage_exceptions.NotFoundError: -raise AriaCliError('Service {0} does not exist'.format(service_name)) +service = model_storage.service.get_by_name(service_name) +filters = dict(service=service) else: logger.info('Listing all executions...') filters = {} http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/38d58d4d/aria/cli/commands/logs.py -- diff --git a/aria/cli/commands/logs.py b/aria/cli/commands/logs.py index 6890fb4..3662063 100644 --- a/aria/cli/commands/logs.py +++ b/aria/cli/commands/logs.py @@ -15,8 +15,6 @@ from .. import utils from ..cli import aria -from ..exceptions import AriaCliError -from ...storage import exceptions as storage_exceptions @aria.group(name='logs') @@ -42,12 +40,8 @@ def list(execution_id, """ logger.info('Listing logs for execution id {0}'.format(execution_id)) # events_logger = get_events_logger(json_output) -try: -logs = model_storage.log.list(filters=dict(execution_fk=execution_id), - sort=utils.storage_sort_param('created_at', False)) -except storage_exceptions.NotFoundError: -raise AriaCliError('Execution {0} does not exist'.format(execution_id)) - +logs = model_storage.log.list(filters=dict(execution_fk=execution_id), + sort=utils.storage_sort_param('created_at', False)) # TODO: print logs nicely if logs: for log in logs: @@ -68,11 +62,7 @@ def delete(execution_id, model_storage, logger): `EXECUTION_ID` is the execution logs to delete. """ logger.info('Deleting logs for execution id {0}'.format(execution_id)) -try: -logs = model_storage.log.list(filters=dict(execution_fk=execution_id)) -except storage_exceptions.NotFoundError: -raise AriaCliError('Execution {0} does not exist'.format(execution_id)) - +logs =
[22/24] incubator-ariatosca git commit: tiny fix to execution error printing
tiny fix to execution error printing Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/6016e146 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/6016e146 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/6016e146 Branch: refs/heads/ARIA-48-aria-cli Commit: 6016e14607279d67c9e061e610fe89225bf4946e Parents: c562426 Author: Ran ZivAuthored: Tue Apr 4 12:07:32 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/executions.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/6016e146/aria/cli/commands/executions.py -- diff --git a/aria/cli/commands/executions.py b/aria/cli/commands/executions.py index 82ee51a..635ae63 100644 --- a/aria/cli/commands/executions.py +++ b/aria/cli/commands/executions.py @@ -150,7 +150,7 @@ def start(workflow_name, execution = workflow_runner.execution logger.info('Execution has ended with "{0}" status'.format(execution.status)) -if execution.status == Execution.FAILED: +if execution.status == Execution.FAILED and execution.error: logger.info('Execution error:\n{0}'.format(execution.error)) if dry:
[01/24] incubator-ariatosca git commit: ARIA-136-ctx-binary-doesnt-get-installed-via-pip-install [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-48-aria-cli 600a9b26c -> 31a0c794f (forced update) ARIA-136-ctx-binary-doesnt-get-installed-via-pip-install Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/369323b6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/369323b6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/369323b6 Branch: refs/heads/ARIA-48-aria-cli Commit: 369323b66cdd02ccfba6cc3e3c547ac4694fc2fd Parents: 2d83475 Author: max-orlovAuthored: Sun Apr 2 19:24:20 2017 +0300 Committer: max-orlov Committed: Tue Apr 4 11:48:08 2017 +0300 -- setup.py | 47 --- 1 file changed, 36 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/369323b6/setup.py -- diff --git a/setup.py b/setup.py index 7be5275..3d72ebc 100644 --- a/setup.py +++ b/setup.py @@ -19,6 +19,7 @@ import sys from setuptools import setup, find_packages from setuptools.command.install import install +from setuptools.command.develop import develop _PACKAGE_NAME = 'aria' _PYTHON_SUPPORTED_VERSIONS = [(2, 6), (2, 7)] @@ -63,20 +64,43 @@ except IOError: console_scripts = ['aria = aria.cli.cli:main'] -class InstallCommand(install): -user_options = install.user_options + [ +def _generate_user_options(command): +return command.user_options + [ ('skip-ctx', None, 'Install with or without the ctx (Defaults to False)') ] -boolean_options = install.boolean_options + ['skip-ctx'] -def initialize_options(self): -install.initialize_options(self) -self.skip_ctx = False -def run(self): -if self.skip_ctx is False: -console_scripts.append('ctx = aria.orchestrator.execution_plugin.ctx_proxy.client:main') -install.run(self) +def _generate_boolean_options(command): +return command.boolean_options + ['skip-ctx'] + + +def _initialize_options(custom_cmd): +custom_cmd.command.initialize_options(custom_cmd) +custom_cmd.skip_ctx = False + + +def _run(custom_cmd): +if custom_cmd.skip_ctx is False: +console_scripts.append('ctx = aria.orchestrator.execution_plugin.ctx_proxy.client:main') +custom_cmd.command.run(custom_cmd) + + +class InstallCommand(install): +command = install + +user_options = _generate_user_options(install) +boolean_options = _generate_boolean_options(install) +initialize_options = _initialize_options +run = _run + + +class DevelopCommand(develop): +command = develop + +user_options = _generate_user_options(develop) +boolean_options = _generate_boolean_options(develop) +initialize_options = _initialize_options +run = _run setup( name=_PACKAGE_NAME, @@ -116,6 +140,7 @@ setup( 'console_scripts': console_scripts }, cmdclass={ -'install': InstallCommand +'install': InstallCommand, # used in pip install ... +'develop': DevelopCommand # used in pip install -e ... } )
[06/24] incubator-ariatosca git commit: fixed storage exceptions in CLI
fixed storage exceptions in CLI Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/c40f3615 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/c40f3615 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/c40f3615 Branch: refs/heads/ARIA-48-aria-cli Commit: c40f3615a21ce83fa98fbeec2c1b617959e7bba2 Parents: 349d4d0 Author: Ran ZivAuthored: Mon Apr 3 14:58:17 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/executions.py | 14 +- aria/cli/commands/logs.py | 16 +--- aria/cli/commands/node_templates.py | 6 +++--- aria/cli/commands/nodes.py | 6 +++--- aria/cli/commands/services.py | 23 ++- aria/storage/exceptions.py | 4 aria/storage/sql_mapi.py| 8 7 files changed, 46 insertions(+), 31 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c40f3615/aria/cli/commands/executions.py -- diff --git a/aria/cli/commands/executions.py b/aria/cli/commands/executions.py index d767fa1..fd47eb3 100644 --- a/aria/cli/commands/executions.py +++ b/aria/cli/commands/executions.py @@ -13,15 +13,12 @@ # See the License for the specific language governing permissions and # limitations under the License. -import json -import time - from .. import utils from ..table import print_data -from ..cli import aria, helptexts +from ..cli import aria from ..exceptions import AriaCliError from ...modeling.models import Execution -from ...storage.exceptions import StorageError +from ...storage import exceptions as storage_exceptions from ...orchestrator.workflow_runner import WorkflowRunner from ...utils import formatting from ...utils import threading @@ -52,7 +49,7 @@ def show(execution_id, model_storage, logger): try: logger.info('Showing execution {0}'.format(execution_id)) execution = model_storage.execution.get(execution_id) -except StorageError: +except storage_exceptions.NotFoundError: raise AriaCliError('Execution {0} not found'.format(execution_id)) print_data(EXECUTION_COLUMNS, execution.to_dict(), 'Execution:', max_width=50) @@ -94,9 +91,8 @@ def list(service_name, try: service = model_storage.service.get_by_name(service_name) filters = dict(service=service) -except StorageError: -raise AriaCliError('Service {0} does not exist'.format( -service_name)) +except storage_exceptions.NotFoundError: +raise AriaCliError('Service {0} does not exist'.format(service_name)) else: logger.info('Listing all executions...') filters = {} http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c40f3615/aria/cli/commands/logs.py -- diff --git a/aria/cli/commands/logs.py b/aria/cli/commands/logs.py index 3662063..6890fb4 100644 --- a/aria/cli/commands/logs.py +++ b/aria/cli/commands/logs.py @@ -15,6 +15,8 @@ from .. import utils from ..cli import aria +from ..exceptions import AriaCliError +from ...storage import exceptions as storage_exceptions @aria.group(name='logs') @@ -40,8 +42,12 @@ def list(execution_id, """ logger.info('Listing logs for execution id {0}'.format(execution_id)) # events_logger = get_events_logger(json_output) -logs = model_storage.log.list(filters=dict(execution_fk=execution_id), - sort=utils.storage_sort_param('created_at', False)) +try: +logs = model_storage.log.list(filters=dict(execution_fk=execution_id), + sort=utils.storage_sort_param('created_at', False)) +except storage_exceptions.NotFoundError: +raise AriaCliError('Execution {0} does not exist'.format(execution_id)) + # TODO: print logs nicely if logs: for log in logs: @@ -62,7 +68,11 @@ def delete(execution_id, model_storage, logger): `EXECUTION_ID` is the execution logs to delete. """ logger.info('Deleting logs for execution id {0}'.format(execution_id)) -logs = model_storage.log.list(filters=dict(execution_fk=execution_id)) +try: +logs = model_storage.log.list(filters=dict(execution_fk=execution_id)) +except storage_exceptions.NotFoundError: +raise AriaCliError('Execution {0} does not exist'.format(execution_id)) + for log in logs: model_storage.log.delete(log) logger.info('Deleted logs for execution id {0}'.format(execution_id))
[11/24] incubator-ariatosca git commit: Enforce Uniqueness on ServiceTemplate and Service names
Enforce Uniqueness on ServiceTemplate and Service names Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/15e2c09b Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/15e2c09b Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/15e2c09b Branch: refs/heads/ARIA-48-aria-cli Commit: 15e2c09b7b28eac348107a81e5fe7a9d379114e4 Parents: c4ca1e8 Author: Avia EfratAuthored: Sun Apr 2 16:37:51 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/service_templates.py | 19 ++- aria/cli/commands/services.py | 7 +++ aria/cli/constants.py | 4 aria/modeling/models.py| 8 ++-- 4 files changed, 31 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/15e2c09b/aria/cli/commands/service_templates.py -- diff --git a/aria/cli/commands/service_templates.py b/aria/cli/commands/service_templates.py index b855529..823a6e8 100644 --- a/aria/cli/commands/service_templates.py +++ b/aria/cli/commands/service_templates.py @@ -19,12 +19,14 @@ import json from .. import utils from .. import csar -from ..cli import aria from .. import service_template_utils +from ..cli import aria +from ..constants import TWO_MODELS_WITH_THE_SAME_NAME_ERROR_TEMPLATE from ..table import print_data from ..exceptions import AriaCliError from ...core import Core from ...exceptions import AriaException +from ...storage.exceptions import StorageError DESCRIPTION_LIMIT = 20 @@ -112,11 +114,18 @@ def store(service_template_path, service_template_name, model_storage, resource_ service_template_path = service_template_utils.get(service_template_path) core = Core(model_storage, resource_storage, plugin_manager) -core.create_service_template(service_template_path, - os.path.dirname(service_template_path), - service_template_name) +try: +core.create_service_template(service_template_path, + os.path.dirname(service_template_path), + service_template_name) +except StorageError: +logger.info(TWO_MODELS_WITH_THE_SAME_NAME_ERROR_TEMPLATE.format( +model_class='service template', +name=service_template_name)) +raise -logger.info('Service template stored') +else: +logger.info('Service template stored') @service_templates.command(name='delete', http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/15e2c09b/aria/cli/commands/services.py -- diff --git a/aria/cli/commands/services.py b/aria/cli/commands/services.py index ce1139b..ae04d7e 100644 --- a/aria/cli/commands/services.py +++ b/aria/cli/commands/services.py @@ -19,11 +19,13 @@ from StringIO import StringIO from . import service_templates from ..cli import aria, helptexts +from ..constants import TWO_MODELS_WITH_THE_SAME_NAME_ERROR_TEMPLATE from ..exceptions import AriaCliError from ..table import print_data from ..utils import storage_sort_param from ...core import Core from ...exceptions import AriaException +from ...storage.exceptions import StorageError SERVICE_COLUMNS = ['id', 'name', 'service_template_name', 'created_at', 'updated_at'] @@ -97,6 +99,11 @@ def create(service_template_name, try: core = Core(model_storage, resource_storage, plugin_manager) service = core.create_service(service_template_name, inputs, service_name) +except StorageError: +logger.info(TWO_MODELS_WITH_THE_SAME_NAME_ERROR_TEMPLATE.format( +model_class='service', +name=service_name)) +raise except AriaException as e: logger.info(str(e)) service_templates.print_service_template_inputs(model_storage, service_template_name) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/15e2c09b/aria/cli/constants.py -- diff --git a/aria/cli/constants.py b/aria/cli/constants.py index 67c094d..fdd37fc 100644 --- a/aria/cli/constants.py +++ b/aria/cli/constants.py @@ -16,3 +16,7 @@ SAMPLE_SERVICE_TEMPLATE_FILENAME = 'service_template.yaml' HELP_TEXT_COLUMN_BUFFER = 5 +TWO_MODELS_WITH_THE_SAME_NAME_ERROR_TEMPLATE = """ +Could not store {model_class} `{name}` +There already a exists a {model_class} with the same name +""" http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/15e2c09b/aria/modeling/models.py
[23/24] incubator-ariatosca git commit: Fix handling creating a service with an existing name
Fix handling creating a service with an existing name Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/31a0c794 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/31a0c794 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/31a0c794 Branch: refs/heads/ARIA-48-aria-cli Commit: 31a0c794f1991342b75f018c0d071efdd128376e Parents: 6016e14 Author: Avia EfratAuthored: Tue Apr 4 12:18:01 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/core.py | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/31a0c794/aria/core.py -- diff --git a/aria/core.py b/aria/core.py index 1fd3cf7..057f716 100644 --- a/aria/core.py +++ b/aria/core.py @@ -21,6 +21,7 @@ from .parser.consumption import ( Validate, ServiceTemplate) from .parser.loading.location import UriLocation +from .storage import exceptions as storage_exceptions class Core(object): @@ -67,6 +68,7 @@ class Core(object): self.resource_storage.service_template.delete(entry_id=str(service_template.id)) def create_service(self, service_template_id, inputs, service_name=None): + service_template = self.model_storage.service_template.get(service_template_id) # creating an empty ConsumptionContext, initiating a threadlocal context @@ -74,8 +76,17 @@ class Core(object): with self.model_storage._all_api_kwargs['session'].no_autoflush: service = service_template.instantiate(None, inputs) -service.name = service_name or '{0}_{1}'.format(service_template.name, service.id) +# If the user didn't enter a name for this service, we'll want to auto generate it. +# But how will we ensure a unique but simple name? We'll append the services' unique id +# to the service_templates name. Since this service is not in the storage yet, we'll put it +# there, and pull out its id. self.model_storage.service.put(service) +service.name = service_name or '{0}_{1}'.format(service_template.name, service.id) +try: +self.model_storage.service.update(service) +except storage_exceptions.StorageError: +self.model_storage.service.delete(service) +raise return service def delete_service(self, service_id, force=False):
[24/24] incubator-ariatosca git commit: few deletion-commands related fixes
few deletion-commands related fixes Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/108d8a69 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/108d8a69 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/108d8a69 Branch: refs/heads/ARIA-48-aria-cli Commit: 108d8a69bb97f44da906cd782fb262929cc22b0f Parents: 84b811c Author: Ran ZivAuthored: Mon Apr 3 23:28:11 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/cli/commands/service_templates.py | 13 +++-- aria/cli/commands/services.py | 3 ++- aria/core.py | 4 ++-- 3 files changed, 11 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/108d8a69/aria/cli/commands/service_templates.py -- diff --git a/aria/cli/commands/service_templates.py b/aria/cli/commands/service_templates.py index 6c6224b..a324131 100644 --- a/aria/cli/commands/service_templates.py +++ b/aria/cli/commands/service_templates.py @@ -126,23 +126,24 @@ def store(service_template_path, service_template_name, model_storage, resource_ @service_templates.command(name='delete', short_help='Delete a service template') -@aria.argument('service-template-id') +@aria.argument('service-template-name') @aria.options.verbose() @aria.pass_model_storage @aria.pass_resource_storage @aria.pass_plugin_manager @aria.pass_logger -def delete(service_template_id, model_storage, resource_storage, plugin_manager, logger): +def delete(service_template_name, model_storage, resource_storage, plugin_manager, logger): """Delete a service template -`SERVICE_TEMPLATE_ID` is the id of the service template to delete. +`SERVICE_TEMPLATE_NAME` is the name of the service template to delete. """ -logger.info('Deleting service template {0}...'.format(service_template_id)) +logger.info('Deleting service template {0}...'.format(service_template_name)) +service_template = model_storage.service_template.get_by_name(service_template_name) core = Core(model_storage, resource_storage, plugin_manager) try: -core.delete_service_template(service_template_id) +core.delete_service_template(service_template.id) except storage_exceptions.NotFoundError: raise AriaCliError() -logger.info('Service template {0} deleted'.format(service_template_id)) +logger.info('Service template {0} deleted'.format(service_template_name)) @service_templates.command(name='inputs', http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/108d8a69/aria/cli/commands/services.py -- diff --git a/aria/cli/commands/services.py b/aria/cli/commands/services.py index 9f9d104..e09db21 100644 --- a/aria/cli/commands/services.py +++ b/aria/cli/commands/services.py @@ -123,8 +123,9 @@ def delete(service_name, force, model_storage, resource_storage, plugin_manager, `SERVICE_NAME` is the name of the service to delete. """ logger.info('Deleting service {0}...'.format(service_name)) +service = model_storage.service.get_by_name(service_name) core = Core(model_storage, resource_storage, plugin_manager) -core.delete_service(service_name, force=force) +core.delete_service(service.id, force=force) logger.info('Service {0} deleted'.format(service_name)) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/108d8a69/aria/core.py -- diff --git a/aria/core.py b/aria/core.py index a589a8e..1fd3cf7 100644 --- a/aria/core.py +++ b/aria/core.py @@ -88,11 +88,11 @@ class Core(object): "Active execution id: {1}".format(service.name, active_executions[0].id)) if not force: -available_nodes = [n for n in service.nodes.values() if n.is_available()] +available_nodes = [str(n.id) for n in service.nodes.values() if n.is_available()] if available_nodes: raise exceptions.DependentAvailableNodesError( "Can't delete service {0} - there are available nodes for this service. " -"Available node ids: {1}".format(service.name, available_nodes)) +"Available node ids: {1}".format(service.name, ', '.join(available_nodes))) self.model_storage.service.delete(service)
[02/24] incubator-ariatosca git commit: ARIA-132-fixed-support-for-cascading-deletion
ARIA-132-fixed-support-for-cascading-deletion Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/e7ffc735 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/e7ffc735 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/e7ffc735 Branch: refs/heads/ARIA-48-aria-cli Commit: e7ffc7353dd2f23fdffde22bdb2ffac20542d601 Parents: 369323b Author: max-orlovAuthored: Tue Apr 4 12:22:12 2017 +0300 Committer: max-orlov Committed: Tue Apr 4 12:22:12 2017 +0300 -- aria/modeling/relationship.py | 3 ++- tests/storage/test_model_storage.py | 13 + 2 files changed, 15 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/e7ffc735/aria/modeling/relationship.py -- diff --git a/aria/modeling/relationship.py b/aria/modeling/relationship.py index ac1de28..e6830b8 100644 --- a/aria/modeling/relationship.py +++ b/aria/modeling/relationship.py @@ -193,7 +193,8 @@ def one_to_many(model_class, child_table, back_populates=back_populates, other_fk=child_fk, -dict_key=dict_key) +dict_key=dict_key, +relationship_kwargs=dict(cascade='all')) def many_to_one(model_class, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/e7ffc735/tests/storage/test_model_storage.py -- diff --git a/tests/storage/test_model_storage.py b/tests/storage/test_model_storage.py index fa57e6f..e4f3eba 100644 --- a/tests/storage/test_model_storage.py +++ b/tests/storage/test_model_storage.py @@ -108,6 +108,19 @@ def test_application_storage_factory(): tests_storage.release_sqlite_storage(storage) +def test_cascade_deletion(context): +service = context.model.service.list()[0] + +assert len(context.model.service_template.list()) == 1 +assert len(service.nodes) == len(context.model.node.list()) == 2 + +context.model.service.delete(service) + +assert len(context.model.service_template.list()) == 1 +assert len(context.model.service.list()) == 0 +assert len(context.model.node.list()) == 0 + + @pytest.fixture def context(tmpdir): result = mock.context.simple(str(tmpdir))
[16/24] incubator-ariatosca git commit: Fix issue in creating services with existing name
Fix issue in creating services with existing name Up until now, creating a service with a name that already existed resulted in a null service stored in the storage. I noticed it while thinking about tests to the `services list` command. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/72c9969c Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/72c9969c Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/72c9969c Branch: refs/heads/ARIA-48-aria-cli Commit: 72c9969c2f4124f59a3f5b787393b1f8bece248d Parents: bf0adb5 Author: Avia EfratAuthored: Sun Apr 2 23:27:55 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 13:20:46 2017 +0300 -- aria/core.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/72c9969c/aria/core.py -- diff --git a/aria/core.py b/aria/core.py index f2dfb09..5368f9c 100644 --- a/aria/core.py +++ b/aria/core.py @@ -80,10 +80,8 @@ class Core(object): service.inputs = modeling_utils.create_inputs(inputs, template_inputs) # TODO: now that we have inputs, we should scan properties and inputs and evaluate functions -# first put the service model so it could have an id, as fallback for setting its name -self.model_storage.service.put(service) service.name = service_name or '{0}_{1}'.format(service_template_name, service.id) -self.model_storage.service.update(service) +self.model_storage.service.put(service) return service def delete_service(self, service_name, force=False):
[jira] [Commented] (ARIA-132) Models cascading deletion raises constraint errors
[ https://issues.apache.org/jira/browse/ARIA-132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954952#comment-15954952 ] ASF subversion and git services commented on ARIA-132: -- Commit e7ffc7353dd2f23fdffde22bdb2ffac20542d601 in incubator-ariatosca's branch refs/heads/master from [~Mxmrlv] [ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=e7ffc73 ] ARIA-132-fixed-support-for-cascading-deletion > Models cascading deletion raises constraint errors > -- > > Key: ARIA-132 > URL: https://issues.apache.org/jira/browse/ARIA-132 > Project: AriaTosca > Issue Type: Bug >Reporter: Ran Ziv >Assignee: Maxim Orlov >Priority: Critical > Fix For: 0.1.0 > > > Models whose deletion should cascade to additional models currently don't > behave this way and instead raise constraint errors -- This message was sent by Atlassian JIRA (v6.3.15#6346)
incubator-ariatosca git commit: ARIA-132-fixed-support-for-cascading-deletion
Repository: incubator-ariatosca Updated Branches: refs/heads/master 369323b66 -> e7ffc7353 ARIA-132-fixed-support-for-cascading-deletion Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/e7ffc735 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/e7ffc735 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/e7ffc735 Branch: refs/heads/master Commit: e7ffc7353dd2f23fdffde22bdb2ffac20542d601 Parents: 369323b Author: max-orlovAuthored: Tue Apr 4 12:22:12 2017 +0300 Committer: max-orlov Committed: Tue Apr 4 12:22:12 2017 +0300 -- aria/modeling/relationship.py | 3 ++- tests/storage/test_model_storage.py | 13 + 2 files changed, 15 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/e7ffc735/aria/modeling/relationship.py -- diff --git a/aria/modeling/relationship.py b/aria/modeling/relationship.py index ac1de28..e6830b8 100644 --- a/aria/modeling/relationship.py +++ b/aria/modeling/relationship.py @@ -193,7 +193,8 @@ def one_to_many(model_class, child_table, back_populates=back_populates, other_fk=child_fk, -dict_key=dict_key) +dict_key=dict_key, +relationship_kwargs=dict(cascade='all')) def many_to_one(model_class, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/e7ffc735/tests/storage/test_model_storage.py -- diff --git a/tests/storage/test_model_storage.py b/tests/storage/test_model_storage.py index fa57e6f..e4f3eba 100644 --- a/tests/storage/test_model_storage.py +++ b/tests/storage/test_model_storage.py @@ -108,6 +108,19 @@ def test_application_storage_factory(): tests_storage.release_sqlite_storage(storage) +def test_cascade_deletion(context): +service = context.model.service.list()[0] + +assert len(context.model.service_template.list()) == 1 +assert len(service.nodes) == len(context.model.node.list()) == 2 + +context.model.service.delete(service) + +assert len(context.model.service_template.list()) == 1 +assert len(context.model.service.list()) == 0 +assert len(context.model.node.list()) == 0 + + @pytest.fixture def context(tmpdir): result = mock.context.simple(str(tmpdir))
[GitHub] incubator-ariatosca pull request #94: ARIA-132-fixed-support-for-cascading-d...
GitHub user mxmrlv opened a pull request: https://github.com/apache/incubator-ariatosca/pull/94 ARIA-132-fixed-support-for-cascading-deletion You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-ariatosca ARIA-132-fixed-support-for-cascading-deletion Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-ariatosca/pull/94.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #94 commit e7ffc7353dd2f23fdffde22bdb2ffac20542d601 Author: max-orlovDate: 2017-04-04T09:22:12Z ARIA-132-fixed-support-for-cascading-deletion --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ARIA-132) Models cascading deletion raises constraint errors
[ https://issues.apache.org/jira/browse/ARIA-132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954889#comment-15954889 ] ASF subversion and git services commented on ARIA-132: -- Commit e7ffc7353dd2f23fdffde22bdb2ffac20542d601 in incubator-ariatosca's branch refs/heads/ARIA-132-fixed-support-for-cascading-deletion from [~Mxmrlv] [ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=e7ffc73 ] ARIA-132-fixed-support-for-cascading-deletion > Models cascading deletion raises constraint errors > -- > > Key: ARIA-132 > URL: https://issues.apache.org/jira/browse/ARIA-132 > Project: AriaTosca > Issue Type: Bug >Reporter: Ran Ziv >Assignee: Maxim Orlov >Priority: Critical > Fix For: 0.1.0 > > > Models whose deletion should cascade to additional models currently don't > behave this way and instead raise constraint errors -- This message was sent by Atlassian JIRA (v6.3.15#6346)
incubator-ariatosca git commit: Fix handling creating a service with an existing name
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-48-aria-cli b3d2e0541 -> 600a9b26c Fix handling creating a service with an existing name Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/600a9b26 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/600a9b26 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/600a9b26 Branch: refs/heads/ARIA-48-aria-cli Commit: 600a9b26c1deb019e24b06210d6cceecbd0dfa36 Parents: b3d2e05 Author: Avia EfratAuthored: Tue Apr 4 12:18:01 2017 +0300 Committer: Avia Efrat Committed: Tue Apr 4 12:18:01 2017 +0300 -- aria/core.py | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/600a9b26/aria/core.py -- diff --git a/aria/core.py b/aria/core.py index 1fd3cf7..057f716 100644 --- a/aria/core.py +++ b/aria/core.py @@ -21,6 +21,7 @@ from .parser.consumption import ( Validate, ServiceTemplate) from .parser.loading.location import UriLocation +from .storage import exceptions as storage_exceptions class Core(object): @@ -67,6 +68,7 @@ class Core(object): self.resource_storage.service_template.delete(entry_id=str(service_template.id)) def create_service(self, service_template_id, inputs, service_name=None): + service_template = self.model_storage.service_template.get(service_template_id) # creating an empty ConsumptionContext, initiating a threadlocal context @@ -74,8 +76,17 @@ class Core(object): with self.model_storage._all_api_kwargs['session'].no_autoflush: service = service_template.instantiate(None, inputs) -service.name = service_name or '{0}_{1}'.format(service_template.name, service.id) +# If the user didn't enter a name for this service, we'll want to auto generate it. +# But how will we ensure a unique but simple name? We'll append the services' unique id +# to the service_templates name. Since this service is not in the storage yet, we'll put it +# there, and pull out its id. self.model_storage.service.put(service) +service.name = service_name or '{0}_{1}'.format(service_template.name, service.id) +try: +self.model_storage.service.update(service) +except storage_exceptions.StorageError: +self.model_storage.service.delete(service) +raise return service def delete_service(self, service_id, force=False):
incubator-ariatosca git commit: tiny fix to execution error printing
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-48-aria-cli 277c4ae82 -> b3d2e0541 tiny fix to execution error printing Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/b3d2e054 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/b3d2e054 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/b3d2e054 Branch: refs/heads/ARIA-48-aria-cli Commit: b3d2e0541fe01fe92876440b15d10e060ab89a6a Parents: 277c4ae Author: Ran ZivAuthored: Tue Apr 4 12:07:32 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 12:07:32 2017 +0300 -- aria/cli/commands/executions.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/b3d2e054/aria/cli/commands/executions.py -- diff --git a/aria/cli/commands/executions.py b/aria/cli/commands/executions.py index 82ee51a..635ae63 100644 --- a/aria/cli/commands/executions.py +++ b/aria/cli/commands/executions.py @@ -150,7 +150,7 @@ def start(workflow_name, execution = workflow_runner.execution logger.info('Execution has ended with "{0}" status'.format(execution.status)) -if execution.status == Execution.FAILED: +if execution.status == Execution.FAILED and execution.error: logger.info('Execution error:\n{0}'.format(execution.error)) if dry:
incubator-ariatosca git commit: created dry-run executor
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-48-aria-cli 4c168db8e -> 277c4ae82 created dry-run executor Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/277c4ae8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/277c4ae8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/277c4ae8 Branch: refs/heads/ARIA-48-aria-cli Commit: 277c4ae82a00833ecb533db18ac694d8727c5fa3 Parents: 4c168db Author: Ran ZivAuthored: Tue Apr 4 12:02:11 2017 +0300 Committer: Ran Ziv Committed: Tue Apr 4 12:02:11 2017 +0300 -- aria/cli/cli/aria.py| 5 +++ aria/cli/cli/helptexts.py | 2 + aria/cli/commands/executions.py | 15 +-- aria/orchestrator/workflow_runner.py| 5 ++- aria/orchestrator/workflows/executor/dry.py | 54 5 files changed, 76 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/277c4ae8/aria/cli/cli/aria.py -- diff --git a/aria/cli/cli/aria.py b/aria/cli/cli/aria.py index baa72eb..1664ce5 100644 --- a/aria/cli/cli/aria.py +++ b/aria/cli/cli/aria.py @@ -304,6 +304,11 @@ class Options(object): is_flag=True, help=helptexts.JSON_OUTPUT) +self.dry_execution = click.option( +'--dry', +is_flag=True, +help=helptexts.DRY_EXECUTION) + self.init_hard_reset = click.option( '--hard', is_flag=True, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/277c4ae8/aria/cli/cli/helptexts.py -- diff --git a/aria/cli/cli/helptexts.py b/aria/cli/cli/helptexts.py index 02519cb..0d66d6b 100644 --- a/aria/cli/cli/helptexts.py +++ b/aria/cli/cli/helptexts.py @@ -32,6 +32,8 @@ HARD_RESET = "Hard reset the configuration, including coloring and loggers" ENABLE_COLORS = "Enable colors in logger (use --hard when working with" \ " an initialized environment) [default: False]" +DRY_EXECUTION = "Execute a workflow dry run (prints operations information without causing side " \ +"effects)" SERVICE_TEMPLATE_FILENAME = ( "The name of the archive's main service template file. " "This is only relevant if uploading an archive") http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/277c4ae8/aria/cli/commands/executions.py -- diff --git a/aria/cli/commands/executions.py b/aria/cli/commands/executions.py index 6d8b949..82ee51a 100644 --- a/aria/cli/commands/executions.py +++ b/aria/cli/commands/executions.py @@ -18,6 +18,7 @@ from ..table import print_data from ..cli import aria from ...modeling.models import Execution from ...orchestrator.workflow_runner import WorkflowRunner +from ...orchestrator.workflows.executor.dry import DryExecutor from ...utils import formatting from ...utils import threading @@ -101,6 +102,7 @@ def list(service_name, @aria.argument('workflow-name') @aria.options.service_name(required=True) @aria.options.inputs +@aria.options.dry_execution @aria.options.task_max_attempts() @aria.options.task_retry_interval() @aria.options.verbose() @@ -111,6 +113,7 @@ def list(service_name, def start(workflow_name, service_name, inputs, + dry, task_max_attempts, task_retry_interval, model_storage, @@ -119,19 +122,21 @@ def start(workflow_name, logger): """Execute a workflow -`WORKFLOW_ID` is the id of the workflow to execute (e.g. `uninstall`) +`WORKFLOW_NAME` is the name of the workflow to execute (e.g. `uninstall`) """ +executor = DryExecutor() if dry else None # use WorkflowRunner's default executor + workflow_runner = \ WorkflowRunner(workflow_name, service_name, inputs, model_storage, resource_storage, plugin_manager, - task_max_attempts, task_retry_interval) + executor, task_max_attempts, task_retry_interval) execution_thread_name = '{0}_{1}'.format(service_name, workflow_name) execution_thread = threading.ExceptionThread(target=workflow_runner.execute, name=execution_thread_name) execution_thread.daemon = True # allows force-cancel to exit immediately -logger.info('Starting execution. Press Ctrl+C cancel') +logger.info('Starting {0}execution. Press Ctrl+C cancel'.format('dry ' if dry else ''))
[jira] [Commented] (ARIA-136) ctx binary doesn't get installed via pip install
[ https://issues.apache.org/jira/browse/ARIA-136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954839#comment-15954839 ] ASF GitHub Bot commented on ARIA-136: - Github user asfgit closed the pull request at: https://github.com/apache/incubator-ariatosca/pull/93 > ctx binary doesn't get installed via pip install > > > Key: ARIA-136 > URL: https://issues.apache.org/jira/browse/ARIA-136 > Project: AriaTosca > Issue Type: Bug >Reporter: Ran Ziv >Assignee: Maxim Orlov >Priority: Minor > Fix For: 0.1.0 > > > the ctx binary is conditionally installed in {{setup.py}}, but it would seem > the {{pip install}} command does not install it as intended when installing > in editable mode ({{-e}} option) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (ARIA-136) ctx binary doesn't get installed via pip install
[ https://issues.apache.org/jira/browse/ARIA-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Orlov closed ARIA-136. Resolution: Fixed Fix Version/s: 0.1.0 > ctx binary doesn't get installed via pip install > > > Key: ARIA-136 > URL: https://issues.apache.org/jira/browse/ARIA-136 > Project: AriaTosca > Issue Type: Bug >Reporter: Ran Ziv >Assignee: Maxim Orlov >Priority: Minor > Fix For: 0.1.0 > > > the ctx binary is conditionally installed in {{setup.py}}, but it would seem > the {{pip install}} command does not install it as intended when installing > in editable mode ({{-e}} option) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[incubator-ariatosca] Git Push Summary
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-136-ctx-binary-doesnt-get-installed-via-pip-install [deleted] 369323b66
incubator-ariatosca git commit: ARIA-136-ctx-binary-doesnt-get-installed-via-pip-install
Repository: incubator-ariatosca Updated Branches: refs/heads/master 2d834753a -> 369323b66 ARIA-136-ctx-binary-doesnt-get-installed-via-pip-install Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/369323b6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/369323b6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/369323b6 Branch: refs/heads/master Commit: 369323b66cdd02ccfba6cc3e3c547ac4694fc2fd Parents: 2d83475 Author: max-orlovAuthored: Sun Apr 2 19:24:20 2017 +0300 Committer: max-orlov Committed: Tue Apr 4 11:48:08 2017 +0300 -- setup.py | 47 --- 1 file changed, 36 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/369323b6/setup.py -- diff --git a/setup.py b/setup.py index 7be5275..3d72ebc 100644 --- a/setup.py +++ b/setup.py @@ -19,6 +19,7 @@ import sys from setuptools import setup, find_packages from setuptools.command.install import install +from setuptools.command.develop import develop _PACKAGE_NAME = 'aria' _PYTHON_SUPPORTED_VERSIONS = [(2, 6), (2, 7)] @@ -63,20 +64,43 @@ except IOError: console_scripts = ['aria = aria.cli.cli:main'] -class InstallCommand(install): -user_options = install.user_options + [ +def _generate_user_options(command): +return command.user_options + [ ('skip-ctx', None, 'Install with or without the ctx (Defaults to False)') ] -boolean_options = install.boolean_options + ['skip-ctx'] -def initialize_options(self): -install.initialize_options(self) -self.skip_ctx = False -def run(self): -if self.skip_ctx is False: -console_scripts.append('ctx = aria.orchestrator.execution_plugin.ctx_proxy.client:main') -install.run(self) +def _generate_boolean_options(command): +return command.boolean_options + ['skip-ctx'] + + +def _initialize_options(custom_cmd): +custom_cmd.command.initialize_options(custom_cmd) +custom_cmd.skip_ctx = False + + +def _run(custom_cmd): +if custom_cmd.skip_ctx is False: +console_scripts.append('ctx = aria.orchestrator.execution_plugin.ctx_proxy.client:main') +custom_cmd.command.run(custom_cmd) + + +class InstallCommand(install): +command = install + +user_options = _generate_user_options(install) +boolean_options = _generate_boolean_options(install) +initialize_options = _initialize_options +run = _run + + +class DevelopCommand(develop): +command = develop + +user_options = _generate_user_options(develop) +boolean_options = _generate_boolean_options(develop) +initialize_options = _initialize_options +run = _run setup( name=_PACKAGE_NAME, @@ -116,6 +140,7 @@ setup( 'console_scripts': console_scripts }, cmdclass={ -'install': InstallCommand +'install': InstallCommand, # used in pip install ... +'develop': DevelopCommand # used in pip install -e ... } )
[GitHub] incubator-ariatosca pull request #93: ARIA-136-ctx-binary-doesnt-get-install...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-ariatosca/pull/93 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---