[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1050472679 ## tools/github/setup.sh: ## @@ -0,0 +1,79 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +#if [[ $TEST_SUITE =~ Dummy ]] Review Comment: Yes you control the test from there. Select "Dummy" to run the dummy test. It is useful when also you enable Ngrok debugging so you get an ssh command to get into the CI build VM to experiment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1050470989 ## tools/github/runDummyKOTests.sh: ## @@ -0,0 +1,26 @@ +#!/usr/bin/env bash Review Comment: removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1050460516 ## tests/src/test/scala/org/apache/openwhisk/core/containerpool/v2/test/ActivationClientProxyTests.scala: ## @@ -15,6 +15,7 @@ * limitations under the License. */ +/* Review Comment: it was an attempt to pass the travis build to merge... but looks like it does not work -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1050363178 ## tools/github/setup.sh: ## @@ -0,0 +1,79 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +#if [[ $TEST_SUITE =~ Dummy ]] Review Comment: it is already there...if you launvh the dummy build and enable ngrok in a few second you get an ssh command to get into the vm with everythingalready setup to build and test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361:
URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049618814
##
tools/github/setup.sh:
##
@@ -0,0 +1,79 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+#if [[ $TEST_SUITE =~ Dummy ]]
+#then echo skipping setup ; exit 0
+#fi
+
+# retries a command for five times and exits with the non-zero exit if even
after
+# the retries the command did not succeed.
+function retry() {
+ local exitcode=0
+ for i in {1..5};
+ do
+exitcode=0
+"$@" && break || exitcode=$? && echo "$i. attempt failed. Will retry
$((5-i)) more times!" && sleep 1;
+ done
+ if [ $exitcode -ne 0 ]; then
+exit $exitcode
+ fi
+}
+
+# setup docker to listen in port 4243
+sudo systemctl stop docker
+sudo sed -i -e 's!/usr/bin/dockerd -H fd://!/usr/bin/dockerd -H
tcp://0.0.0.0:4243 -H fd://!' /lib/systemd/system/docker.service
Review Comment:
I tried to run tests without this change but some tests (coverage tests)
fails.
So I would keep the open door for Docker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049281941 ## tools/github/runDummyKOTests.sh: ## @@ -0,0 +1,26 @@ +#!/usr/bin/env bash Review Comment: yes to test a failing build... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361:
URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049281580
##
tools/github/setup.sh:
##
@@ -0,0 +1,79 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+#if [[ $TEST_SUITE =~ Dummy ]]
+#then echo skipping setup ; exit 0
+#fi
+
+# retries a command for five times and exits with the non-zero exit if even
after
+# the retries the command did not succeed.
+function retry() {
+ local exitcode=0
+ for i in {1..5};
+ do
+exitcode=0
+"$@" && break || exitcode=$? && echo "$i. attempt failed. Will retry
$((5-i)) more times!" && sleep 1;
+ done
+ if [ $exitcode -ne 0 ]; then
+exit $exitcode
+ fi
+}
+
+# setup docker to listen in port 4243
+sudo systemctl stop docker
+sudo sed -i -e 's!/usr/bin/dockerd -H fd://!/usr/bin/dockerd -H
tcp://0.0.0.0:4243 -H fd://!' /lib/systemd/system/docker.service
Review Comment:
At least in the 1.0.0 branch one test fails... Let me see if we can get rid
of this on the master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049279967 ## tools/github/setup.sh: ## @@ -0,0 +1,79 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +#if [[ $TEST_SUITE =~ Dummy ]] Review Comment: no, if you do not want to defeat the goal of the dummy test to avoid to run a fast build withou waiting an hour... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049279581 ## tools/github/runDummyTests.sh: ## @@ -0,0 +1,26 @@ +#!/usr/bin/env bash Review Comment: to test the build without having to run a full test... it is handy to debug secrets and so on -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049279054 ## .github/workflows/README.md: ## @@ -0,0 +1,56 @@ +# How to use those workflows + +There are a few [GitHub secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets) to configure to fully leverage the build. + +You can use and set the followings secrets also in your fork. + +## Ngrok Debugging + +You can debug a GitHub Action build using [NGROK](https://ngrok.com/). + +It is disabled for automated build triggered by push and pull_requests. + +You can trigger a workflow run manually enabling ngrok debugging. + +It will open an ssh connection to the VM and keep it up and running for one hour. +The connection url is showns in the log for debugAction.sh + +You can then connect to the build vm, and debug it. +You need to use a password of your choice to access it. + +You can continue the build with `touch /tmp/continue`. +You can abort the build with `touch /tmp/abort`. + +To enable this option you have to register to Ngrok, using the fee account and get the NGROK Token. + +Then set the following secrets: + +- `NGROK_TOKEN` to the ngrok token. +- `NGROK_PASSWORD` to a password of choice to access the build with the ssh command generated. + +## Log Upload + +The build uploads the logs to an s3 bucket allowing to inspect them with a browser. + +You need to create the bucket with the following commands: + +``` +AWS_BUCKET= +AWS_REGION= +aws s3 mb s3://$AWS_BUCKET --region $AWS_REGION +aws s3 website s3://$AWS_BUCKET/ --index-document index.html +aws s3api put-bucket-acl --acl public-read --bucket $AWS_BUCKET +``` + +To enable upload to the created bucket you need to set the following secrets: + +- `AWS_BUCKET`: name of your bucket in s3 (just the name, without `s3://`); create it before. +- `AWS_ACCESS_KEY_ID`: your aws access key. +- `AWS_SECRET_ACCESS_KEY`: your aws secret key. +- `AWS_REGION`: important: the region where your bucket is. + +## Slack notification + +If you want to get notified of what happens on slack, create an [Incoming Web Hook](https://api.slack.com/messaging/webhooks) and then set the following secret: + +- `SLACK_WEBHOOK`: the incoming webhook url provided by slack. Review Comment: You can use it with the current OpenWhisk slack. I do not know who has access to the slack configuration. All you need to do is to create an app and retrieve the incoming webhook. And you can use the exiting dev channel on OpenWhisk slack -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361:
URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049270136
##
.github/workflows/0-on-demand.yaml:
##
@@ -0,0 +1,94 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+name: On Demand Tests
+
+on:
+ workflow_dispatch:
+inputs:
+ enable_ngrok_debug:
+description: "Enable Ngrok Debugging"
+required: true
+type: boolean
+default: false
+ test_suite:
+description: Select Test Suite to run
+type: choice
+options:
+- Unit
+- System
+- MultiRuntime
+- Standalone
+- Scheduler
+- Performance
+## Uncomment those to debug the build
+- Dummy
+#- DummyKO
+
+env:
+ # openwhisk env
+ TEST_SUITE: ${{ inputs.test_suite }}
+ ANSIBLE_CMD: "ansible-playbook -i environments/local -e
docker_image_prefix=testing"
+ GRADLE_PROJS_SKIP: ""
+
+ # (optional) slack incoming wehbook for notifications
+ SLACK_WEBHOOK: ${{secrets.SLACK_WEBHOOK}}
+
+ # (optional) s3 log upload
+ AWS_BUCKET: ${{ secrets.AWS_BUCKET }}
Review Comment:
Yes you can put your own AWS_BUCKET where you want. The bucket can be made
public as I did here:
https://nuvolaris-logs.s3.eu-central-1.amazonaws.com/index.html.
Instructions are in the README.
Actually, since there is already a bucket configured for some tests, it will
be used!
Maybe this is not what the owner of the bucket wants.
We need to get a bucket from Amazon or from some other cloud provider, I am
sure there are donations somewhere of resources that we can use...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1049267204 ## .github/workflows/0-on-demand.yaml: ## @@ -0,0 +1,94 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +name: On Demand Tests + +on: + workflow_dispatch: +inputs: + enable_ngrok_debug: Review Comment: The NGROK support is now only in the "on demand" task that must be manually activated. So you trigger it when you want. Works in your fork, so you have to add the token only for your it. When it is enabled in the actions a menu to trigger manually appears, and you have to check the box and select the test you want to run as below.  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361:
URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1048119663
##
ansible/group_vars/all:
##
@@ -458,98 +420,3 @@ metrics:
port: "{{ metrics_kamon_statsd_port | default('8125') }}"
user_events: "{{ user_events_enabled | default(false) | lower }}"
-
-zeroDowntimeDeployment:
Review Comment:
ooops - reverting them to the master version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action
msciabarra commented on code in PR #5361:
URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1046228883
##
ansible/group_vars/all:
##
@@ -551,5 +551,3 @@ scheduler:
throttlingFraction: "{{ scheduler_queue_throttlingFraction | default(0.9)
}}"
durationBufferSize: "{{ scheduler_queue_durationBufferSize | default(10)
}}"
deployment_ignore_error: "{{ scheduler_deployment_ignore_error |
default('False') }}"
- dataManagementService:
Review Comment:
Yes. It gives a duplicate warning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action [WIP]
msciabarra commented on code in PR #5361:
URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1045018310
##
.github/workflows/performance.yaml:
##
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+name: Performance Tests
+on:
+ push:
+
+env:
+ # uncomment this to enable an ngrok access to the vm on test failure
+ # enabling this you will get an ssh command to access the vm, active for one
hour
+ #NGROK_DEBUG: yes
+ # you need to add as secrets an ngrok token and a password to access the
terminal
+ NGROK_TOKEN: ${{ secrets.NGROK_TOKEN }}
Review Comment:
Yes but the NGROK account is free for one connection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [openwhisk] msciabarra commented on a diff in pull request #5361: GitHub action [WIP]
msciabarra commented on code in PR #5361: URL: https://github.com/apache/openwhisk/pull/5361#discussion_r1045018230 ## .github/workflows/unit.yaml: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +name: Unit Tests +on: + push: + +env: + # uncomment this to enable an ngrok access to the vm on test failure + # enabling this you will get an ssh command to access the vm, active for one hour + #NGROK_DEBUG: yes Review Comment: Yes. I already made it optional and you can enable it triggering it explicitly. You can run it on your fork. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@openwhisk.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
