[2/2] incubator-ariatosca git commit: ARIA-46 Execution plugin

2017-01-09 Thread dankilman 
ARIA-46 Execution plugin


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/e203f6ce
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/e203f6ce
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/e203f6ce

Branch: refs/heads/ARIA-46-execution-plugin
Commit: e203f6ce9b3a3221589c0284b8b5532c622d97bf
Parents: 3caf177
Author: Dan Kilman 
Authored: Tue Jan 3 17:00:46 2017 +0200
Committer: Dan Kilman 
Committed: Mon Jan 9 19:17:29 2017 +0200

--
 MANIFEST.in |   1 +
 aria/orchestrator/execution_plugin/__init__.py  |  33 ++
 aria/orchestrator/execution_plugin/common.py| 150 +
 aria/orchestrator/execution_plugin/constants.py |  51 ++
 .../execution_plugin/ctx_proxy/__init__.py  |  16 +
 .../execution_plugin/ctx_proxy/client.py| 105 
 .../execution_plugin/ctx_proxy/server.py| 240 
 .../execution_plugin/environment_globals.py |  60 ++
 .../orchestrator/execution_plugin/exceptions.py |  38 ++
 aria/orchestrator/execution_plugin/local.py | 123 
 .../orchestrator/execution_plugin/operations.py |  59 ++
 .../execution_plugin/ssh/__init__.py|  14 +
 .../execution_plugin/ssh/operations.py  | 190 ++
 .../orchestrator/execution_plugin/ssh/tunnel.py |  91 +++
 aria/orchestrator/workflows/api/task.py |  12 +-
 aria/orchestrator/workflows/core/engine.py  |   1 -
 aria/orchestrator/workflows/core/task.py|   3 +-
 aria/orchestrator/workflows/executor/process.py |   3 +
 aria/storage/base_model.py  |  26 +-
 aria/storage/filesystem_rapi.py |   2 +-
 requirements.txt|   3 +
 setup.py|   9 +-
 tests/orchestrator/execution_plugin/__init__.py |  14 +
 .../execution_plugin/test_common.py | 193 ++
 .../execution_plugin/test_ctx_proxy_server.py   | 355 +++
 .../orchestrator/execution_plugin/test_local.py | 586 +++
 tests/orchestrator/execution_plugin/test_ssh.py | 545 +
 27 files changed, 2910 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/e203f6ce/MANIFEST.in
--
diff --git a/MANIFEST.in b/MANIFEST.in
index f9bd145..6b08eeb 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1 +1,2 @@
 include requirements.txt
+include aria/orchestrator/execution_plugin/ctx_proxy/binaries/ctx

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/e203f6ce/aria/orchestrator/execution_plugin/__init__.py
--
diff --git a/aria/orchestrator/execution_plugin/__init__.py 
b/aria/orchestrator/execution_plugin/__init__.py
new file mode 100644
index 000..372022f
--- /dev/null
+++ b/aria/orchestrator/execution_plugin/__init__.py
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from contextlib import contextmanager
+
+# Populated during execution of python scripts
+ctx = None
+inputs = None
+
+
+@contextmanager
+def python_script_scope(operation_ctx, operation_inputs):
+global ctx
+global inputs
+try:
+ctx = operation_ctx
+inputs = operation_inputs
+yield
+finally:
+ctx = None
+inputs = None

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/e203f6ce/aria/orchestrator/execution_plugin/common.py
--
diff --git a/aria/orchestrator/execution_plugin/common.py 
b/aria/orchestrator/execution_plugin/common.py
new file mode 100644
index 000..c13555a
--- /dev/null
+++ b/aria/orchestrator/execution_plugin/common.py
@@ -0,0 +1,150 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for

[jira] [Commented] (ARIA-46) Execution plugin in ARIA

2017-01-09 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARIA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812293#comment-15812293
 ] 

ASF subversion and git services commented on ARIA-46:
-

Commit e203f6ce9b3a3221589c0284b8b5532c622d97bf in incubator-ariatosca's branch 
refs/heads/ARIA-46-execution-plugin from [~d...@gigaspaces.com]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-ariatosca.git;h=e203f6c ]

ARIA-46 Execution plugin


> Execution plugin in ARIA
> 
>
> Key: ARIA-46
> URL: https://issues.apache.org/jira/browse/ARIA-46
> Project: AriaTosca
>  Issue Type: Story
>Reporter: Ran Ziv
>Assignee: Dan Kilman
>Priority: Critical
>
> Migrate the execution plugin to ARIA, and make it part of the core code



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [DISCUSS] Website

2017-01-09 Thread Ran Ziv 
do we? (want to use jek)

On Mon, Jan 9, 2017 at 2:50 PM, Arthur Berezin 
wrote:

> John, Suneel,  We would like to migrate the website to Jekyll, where can I
> find more information on what servers the web pages from the ASF repos? Is
> there anyone that could help with the migration process, setting up needed
> asf infrastructure?
>
>
> On Mon, Dec 5, 2016 at 2:39 PM Arthur Berezin 
> wrote:
>
> > Hi All,
> > I would like to follow up on this, and pick up the ball for updating
> > ARIA's website according to ASF guidelines.
> >
> > [comments inline]
> >
> > On Mon, Nov 14, 2016 at 4:43 PM John D. Ament 
> > wrote:
> >
> > All,
> >
> > AriaTosca has been incubating since August and still does not have a
> > website.  IMHO this is a potential problem, especially around community
> > building.
> >
> >
> > One idea may be to use the newly crafted Apache Website Template to build
> > the website.  You can see the template here:
> > https://github.com/apache/apache-website-template - its based on jekyll.
> >
> > The flipside, there's already http://ariatosca.org/ which I'm assuming
> is
> > this podling.  It may make sense for this website to become our website,
> > focused on the underlying Apache community.
> >
> >
> > You are correct, www.ariatosca.org is the website we brought up for the
> > ARIA TOSCA Podling,it was brought up before it was accepted as an asf
> > Incubator project. I would like to make sure we are compliant with ASF
> > process as well, and we have all needed information there.
> >
> > Can we use this this as a starting point, add the missing information
> > expected to be there?
> > I've highlighted the gaps we should complete, based on the the Podling
> > webisite guildeline:
> > http://incubator.apache.org/guides/sites.html
> >
> >
> >- Project Home Page: the primary entry point to the site; contains
> >project description, news, invitation to join the project.
> >- License Page: usually, the Apache License 2.0
> >- Downloads: many projects in incubation will release code, and this
> >page describes them and has links to the download pages that redirect
> to
> >Apache Mirror sites.
> >- Documentation: this page describes the project documentation,
> >including javadoc for Java projects; guides, tutorials, and links to
> >external documentation.
> >- *Committers: *a list of current committers on the project.
> >- *Mailing Lists:* there are several mailing lists that the community
> >might be interested in, and this page contains mailto: links that
> >allow easy subscription (and unsubscription) to any of them.
> >- *FAQ: *frequently asked questions are answered here.
> >- Road Map: if the project has a vision of future community or
> >development activities, the road map is published here.
> >- *Source Code: *links to the browsable source repository and svn
> >commands to check out the sources.
> >- Coding Standards: the coding standards for submitted code by the
> >community, along with a description of how strict the project intends
> to be.
> >- *Issue Tracking: *links to the JIRA or other issue tracking tool,
> >possibly including frequently used filters for issue lists.
> >- Dependencies: other projects that this project depends on.
> >- favicon: the project's icon in a format suitable for a browser's
> >address bar. If absent, an Apache Feather will be displayed.
> >
> >
> >
> >
> > Once we have that ironed our, we need to figure out a solution to getting
> > the website published.  The current recommended solution is gitpubsub.
> >
> > It
> > involves setting up a buildbot to monitor this repo to rebuild the
> asf-site
> > branch, then serving that asf-site branch as the actual website.  Its
> > similar to how github pages work, except you get an apache domain.
> >
> >
> > Currently www.ARIATOSCA.org is a wordpress app running on AWS. The
> reason
> > wordpress was chosen was due the low effort and speed we chould bring up
> a
> > graphically designed website, and add content quickly as we go along.
> >
> > John, Suneel, what would be the recomendation here? Should we migrate to
> > Jekyll?
> >
> >
> > Arthur
> >
> >
> >
> > John
> >
> >
>

Re: [DISCUSS] Website

2017-01-09 Thread John D. Ament 
Arthur,

There's some basic docs up on http://apache.org/dev/#web
It lists CMS as the preferred approach.  CMS isn't the preferred any
longer, so this needs some updating.

The recommendation is to use a dedicated git repo for the website code.
Once that's in, you would setup a jenkins job that clones the repo, builds
the site, and then pushes to asf-site.

There's a template for a shell script that does this here:
http://svn.apache.org/viewvc/incubator/public/trunk/build_site.sh?revision=1776892=markup

We should be able to keep the discussions on this list, we can raise infra
tickets as needed.

John

On Mon, Jan 9, 2017 at 7:50 AM Arthur Berezin  wrote:

> John, Suneel,  We would like to migrate the website to Jekyll, where can I
> find more information on what servers the web pages from the ASF repos? Is
> there anyone that could help with the migration process, setting up needed
> asf infrastructure?
>
>
> On Mon, Dec 5, 2016 at 2:39 PM Arthur Berezin 
> wrote:
>
> > Hi All,
> > I would like to follow up on this, and pick up the ball for updating
> > ARIA's website according to ASF guidelines.
> >
> > [comments inline]
> >
> > On Mon, Nov 14, 2016 at 4:43 PM John D. Ament 
> > wrote:
> >
> > All,
> >
> > AriaTosca has been incubating since August and still does not have a
> > website.  IMHO this is a potential problem, especially around community
> > building.
> >
> >
> > One idea may be to use the newly crafted Apache Website Template to build
> > the website.  You can see the template here:
> > https://github.com/apache/apache-website-template - its based on jekyll.
> >
> > The flipside, there's already http://ariatosca.org/ which I'm assuming
> is
> > this podling.  It may make sense for this website to become our website,
> > focused on the underlying Apache community.
> >
> >
> > You are correct, www.ariatosca.org is the website we brought up for the
> > ARIA TOSCA Podling,it was brought up before it was accepted as an asf
> > Incubator project. I would like to make sure we are compliant with ASF
> > process as well, and we have all needed information there.
> >
> > Can we use this this as a starting point, add the missing information
> > expected to be there?
> > I've highlighted the gaps we should complete, based on the the Podling
> > webisite guildeline:
> > http://incubator.apache.org/guides/sites.html
> >
> >
> >- Project Home Page: the primary entry point to the site; contains
> >project description, news, invitation to join the project.
> >- License Page: usually, the Apache License 2.0
> >- Downloads: many projects in incubation will release code, and this
> >page describes them and has links to the download pages that redirect
> to
> >Apache Mirror sites.
> >- Documentation: this page describes the project documentation,
> >including javadoc for Java projects; guides, tutorials, and links to
> >external documentation.
> >- *Committers: *a list of current committers on the project.
> >- *Mailing Lists:* there are several mailing lists that the community
> >might be interested in, and this page contains mailto: links that
> >allow easy subscription (and unsubscription) to any of them.
> >- *FAQ: *frequently asked questions are answered here.
> >- Road Map: if the project has a vision of future community or
> >development activities, the road map is published here.
> >- *Source Code: *links to the browsable source repository and svn
> >commands to check out the sources.
> >- Coding Standards: the coding standards for submitted code by the
> >community, along with a description of how strict the project intends
> to be.
> >- *Issue Tracking: *links to the JIRA or other issue tracking tool,
> >possibly including frequently used filters for issue lists.
> >- Dependencies: other projects that this project depends on.
> >- favicon: the project's icon in a format suitable for a browser's
> >address bar. If absent, an Apache Feather will be displayed.
> >
> >
> >
> >
> > Once we have that ironed our, we need to figure out a solution to getting
> > the website published.  The current recommended solution is gitpubsub.
> >
> > It
> > involves setting up a buildbot to monitor this repo to rebuild the
> asf-site
> > branch, then serving that asf-site branch as the actual website.  Its
> > similar to how github pages work, except you get an apache domain.
> >
> >
> > Currently www.ARIATOSCA.org is a wordpress app running on AWS. The
> reason
> > wordpress was chosen was due the low effort and speed we chould bring up
> a
> > graphically designed website, and add content quickly as we go along.
> >
> > John, Suneel, what would be the recomendation here? Should we migrate to
> > Jekyll?
> >
> >
> > Arthur
> >
> >
> >
> > John
> >
> >
>

Re: [DISCUSS] Website

2017-01-09 Thread Arthur Berezin 
John, Suneel,  We would like to migrate the website to Jekyll, where can I
find more information on what servers the web pages from the ASF repos? Is
there anyone that could help with the migration process, setting up needed
asf infrastructure?


On Mon, Dec 5, 2016 at 2:39 PM Arthur Berezin  wrote:

> Hi All,
> I would like to follow up on this, and pick up the ball for updating
> ARIA's website according to ASF guidelines.
>
> [comments inline]
>
> On Mon, Nov 14, 2016 at 4:43 PM John D. Ament 
> wrote:
>
> All,
>
> AriaTosca has been incubating since August and still does not have a
> website.  IMHO this is a potential problem, especially around community
> building.
>
>
> One idea may be to use the newly crafted Apache Website Template to build
> the website.  You can see the template here:
> https://github.com/apache/apache-website-template - its based on jekyll.
>
> The flipside, there's already http://ariatosca.org/ which I'm assuming is
> this podling.  It may make sense for this website to become our website,
> focused on the underlying Apache community.
>
>
> You are correct, www.ariatosca.org is the website we brought up for the
> ARIA TOSCA Podling,it was brought up before it was accepted as an asf
> Incubator project. I would like to make sure we are compliant with ASF
> process as well, and we have all needed information there.
>
> Can we use this this as a starting point, add the missing information
> expected to be there?
> I've highlighted the gaps we should complete, based on the the Podling
> webisite guildeline:
> http://incubator.apache.org/guides/sites.html
>
>
>- Project Home Page: the primary entry point to the site; contains
>project description, news, invitation to join the project.
>- License Page: usually, the Apache License 2.0
>- Downloads: many projects in incubation will release code, and this
>page describes them and has links to the download pages that redirect to
>Apache Mirror sites.
>- Documentation: this page describes the project documentation,
>including javadoc for Java projects; guides, tutorials, and links to
>external documentation.
>- *Committers: *a list of current committers on the project.
>- *Mailing Lists:* there are several mailing lists that the community
>might be interested in, and this page contains mailto: links that
>allow easy subscription (and unsubscription) to any of them.
>- *FAQ: *frequently asked questions are answered here.
>- Road Map: if the project has a vision of future community or
>development activities, the road map is published here.
>- *Source Code: *links to the browsable source repository and svn
>commands to check out the sources.
>- Coding Standards: the coding standards for submitted code by the
>community, along with a description of how strict the project intends to 
> be.
>- *Issue Tracking: *links to the JIRA or other issue tracking tool,
>possibly including frequently used filters for issue lists.
>- Dependencies: other projects that this project depends on.
>- favicon: the project's icon in a format suitable for a browser's
>address bar. If absent, an Apache Feather will be displayed.
>
>
>
>
> Once we have that ironed our, we need to figure out a solution to getting
> the website published.  The current recommended solution is gitpubsub.
>
> It
> involves setting up a buildbot to monitor this repo to rebuild the asf-site
> branch, then serving that asf-site branch as the actual website.  Its
> similar to how github pages work, except you get an apache domain.
>
>
> Currently www.ARIATOSCA.org is a wordpress app running on AWS. The reason
> wordpress was chosen was due the low effort and speed we chould bring up a
> graphically designed website, and add content quickly as we go along.
>
> John, Suneel, what would be the recomendation here? Should we migrate to
> Jekyll?
>
>
> Arthur
>
>
>
> John
>
>