Hi Michele I did a lot of cleanup recently to make the common tests easier to write. I'm afraid that without a common test suite, we will not have consistency between the rutimes and in particular how they handle some of the standard flows. In doing this normalization recently, I found that some of the runtimes didn't handle certain error conditions the same way, or didn't support the same features (bugs, namely can't handle large inputs consistently).
An explanation of the runtime unit tests is here: https://github.com/apache/incubator-openwhisk/blob/master/docs/actions-new.md#testing-the-new-runtime I am happy to discuss improvements, further extending the test coverage, etc. but we need to make sure all the runtimes pass a minimum bar so that we can reduce maintenance overhead and OpenWhisk users have a consistent experience. Please take a look at these examples of the new tests adopted in the runtimes nodejs: https://github.com/apache/incubator-openwhisk-runtime-nodejs/blob/master/tests/src/test/scala/runtime/actionContainers/NodeJsActionContainerTests.scala#L38-L111 python: https://github.com/apache/incubator-openwhisk-runtime-python/blob/master/tests/src/test/scala/runtime/actionContainers/PythonActionContainerTests.scala#L48-L115 ruby: https://github.com/apache/incubator-openwhisk-runtime-ruby/blob/master/tests/src/test/scala/actionContainers/Ruby25ActionContainerTests.scala#L45-L121 java: https://github.com/apache/incubator-openwhisk-runtime-java/blob/master/tests/src/test/scala/actionContainers/JavaActionContainerTests.scala#L37-L140 php: https://github.com/apache/incubator-openwhisk-runtime-php/blob/master/tests/src/test/scala/runtime/actionContainers/Php7ActionContainerTests.scala#L43-L130 ballerina: https://github.com/apache/incubator-openwhisk-runtime-ballerina/blob/master/tests/src/test/scala/actionContainers/BallerinaActionContainerTests.scala#L43-L74 Of course we should discuss better ways to normalize the runtimes and the user experience around them. -r On Mon, Jul 23, 2018 at 5:13 PM, Michele Sciabarra <[email protected]> wrote: > Yup, but the problem is: do you want to add "scala tests" (and > scalacode) for a "go runtime"? > > I mean, I read the test code and was not that simple. There were a number > of classes involved, and if I remember well some classes were also using > utilities in the main codebase. > > Adding a whole scala build just to run tests when there are already unit > tests in go and an integration test done with a command line tool (cram) it > just enough IMHO. > > I am not going to add Scala Tests to the runtime. Instead I submitted my > PR to use cram for integration test. > > To me, you can forget the ScalaTests, that are a basically a port of the > Cram tests I did for integration in the main repo. > Now, they are no more necessary. > > > -- > Michele Sciabarra > [email protected] > > ----- Original message ----- > From: Rodric Rabbah <[email protected]> > To: [email protected] > Subject: Re: Where to contribute examples for Golang? > Date: Mon, 23 Jul 2018 16:54:44 +0000 > > I vaguely recall it but I can't find the PR, in any case, here's the > rationale for moving the tests to their respective runtime. > > The openwhisk core (specifically, the invoker protocol) is not language > specific. So while we've had many tests for different languages in the > openwhisk repo, we didn't have consistent test suites, there was lots of > clone and own, and too many touch points for adding a new language runtime. > > In recent PRs and housekeeping, we've removed all the runtime specific > tests (REST, wsk CLI, and runtime tests) from the openwhisk repo. Instead, > there's a canonical test suite that aims to enforce some uniformity between > the various runtimes ("unit tests" against the runtime proxy), and a sniff > test as part of the system package that invokes a standard action for each > of the runtimes listed in the manifest. This reduces the touch points in > the openwhisk repo for a new runtime, and localizes all the runtime work to > its own repo and its maintainers. > > I've updated several of the runtimes to inherit this test suite and can > send a PR for the go runtime to do the same. In this way, these tests would > belong in the new go runtime proxy: > tests/src/test/scala/actionContainers/ActionLoopContainerTests.scala > tests/src/test/scala/actionContainers/ActionLoopGoContainerTests.scala > tests/src/test/scala/actionContainers/ActionLoopSwiftContainerTests.scala > > since they're testing the functionality of the proxy. If you've got these > tests covered by go unit tests, then we don't need duplicate tests, I'd > say. > > The dependence on the openwhisk repo for the test harness is managed > through a snapshot build for now (for local development, thanks to Chetan), > and this is how we've set up Travis too. > > -r > > > > On Mon, Jul 23, 2018 at 4:28 PM, Michele Sciabarra < > [email protected]> > wrote: > > > You closed the PR (I have the email somewhere) with a comment "move in > the > > runtime" and I removed the repo as it was only holding that change. > > > > I just pushed again the code on github. It is here: > > > > https://github.com/sciabarracom/incubator-openwhisk > > > > I do not think it is mergeable with the current repo. > > > > The significant files are just: > > > > tests/src/test/scala/actionContainers/ActionLoopContainerTests.scala > > tests/src/test/scala/actionContainers/ActionLoopGoContainerTests.scala > > tests/src/test/scala/actionContainers/ActionLoopSwiftContainerTests. > scala > > tests/src/test/scala/actionContainers/ResourceHelpers.scala > > > > However, since I was requested to move in the repo, I fixed my original, > > "cram based" test. > > > > https://github.com/apache/incubator-openwhisk-runtime-go/pull/43 > > > > I do not really see a good fit to add a lot of scala code to a go repo > > just to run a bunch of simple tests while I already have those tests > > running with Cram (https://pypi.org/project/cram/). Those are really > > just integration tests. > > > > Most of the tests are actually already rewritten in Go and integrated in > > the travis build (it was hard enough as the timing in Travis are > different > > than on a mac). > > > > -- > > Michele Sciabarra > > [email protected] > > > > ----- Original message ----- > > From: Rodric Rabbah <[email protected]> > > To: [email protected] > > Subject: Re: Where to contribute examples for Golang? > > Date: Mon, 23 Jul 2018 15:53:28 +0000 > > > > > I contributed ScalaTest tests for my runtime to the main repository, > as > > it was requested, and my PR was closed because now they should be moved > ... > > in the runtime. > > > > Do you have a link for the PR? This query doesn't return anything > > https://github.com/apache/incubator-openwhisk/pulls? > > utf8=%E2%9C%93&q=is%3Apr+from%3Asciabarra > > maybe I have your github id wrong. I'm happy to take a look and help. > > > > -r > > >
