RE: Dangers of renaming and removing runtime kinds
Hello Tyson, I think your proposal makes perfect sense. It would be very helpful for adopters to have a kind of operator's guide. Most of the documentation is very developer centric and we see a lot of questions in the Openwhisk Slack asking for guidance when deploying the system. There are "beginnings" of a deployer's guide that could be renamed / extended [1]. [1] https://github.com/apache/openwhisk/blob/master/docs/deploy.md Mit freundlichen Grüßen / Regards, Sven Lange-Last Senior Software Engineer IBM Cloud Functions Apache OpenWhisk E-mail: sven.lange-l...@de.ibm.com Find me on: Schoenaicher Str. 220 Boeblingen, 71032 Germany IBM Deutschland Research & Development GmbH Vorsitzende des Aufsichtsrats: Martina Koederitz Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
Re: Dangers of renaming and removing runtime kinds
I think the PR is a good start, but I also think upgrading OpenWhisk is a
hazardous area that has broader scope that operators need to consider when
running OpenWhisk with any custom configuration (and nobody should probably run
with default config, but I guess technically it would be possible).
Considerations include:
* which custom invoker/controller akka configs may conflict with new config
defaults?
* runtime manifest changes or runtime changes (if you use public runtimes)
* kafka message schema changes
* docker image command/args changes that may surface if you have a custom
deployment to run public images
* db changes (I guess less likely an issue since most operators would not run
"custom schemas" on the db?)
I'm not sure how to produce an exhaustive list, but it may be something we
should add to an "upgrading openwhisk" section of an "operators guide" - I
don't know if such a guide currently exists.
Thanks
Tyson
On 9/16/19, 11:43 PM, "Sven Lange-Last" wrote:
Hello Dave,
I absolutely agree that all adopters running Apache Openwhisk as a private
or public production offering will or even should have their own runtimes
manifest - like we do in IBM.
At the same time, we are using the Apache Openwhisk test suite to run
against our IBM version of the system. When action kinds change in this
test suite ("java" to "java:8"), this requires some work on our side. I
admit that's our problem.
With my proposal to improve documentation, I wanted to make adopters aware
of what runtime changes mean. Even if adopters have their own version of
the runtimes manifest, I guess they start with a copy of the Apache
Openwhisk default manifest. So when they set up their runtime manifest,
they hopefully keep the new description to make maintainers of the file
aware that removal of runtime kinds needs to be planned carefully.
Mit freundlichen Grüßen / Regards,
Sven Lange-Last
Senior Software Engineer
IBM Cloud Functions
Apache OpenWhisk
Re: Dangers of renaming and removing runtime kinds
I don't think there is actually a distinction between the two paths in deserialize(). The try path throws the exception inside docReader.read() whereas in the catch, the exception is deferred to the actual type check that occurs on lines 64-67. The exceptions should arguably be the same - I suspect we can eliminate the try/catch (caveat: it's been a while since I looked at that code carefully). The reason the deserializer is the way it is, and the order matters, is that the type of the record is not recorded in the document and so the deserializer relies on schema matches to deserializes a given document. An action and a trigger are similar in schema - if you eliminate the exec property from the former. Perhaps the db interface should address that too (i.e., record the type in the document since by default there is only one db for all assets). -r On Mon, Sep 16, 2019 at 1:51 PM Sven Lange-Last wrote: > Hello Openwhisk community members, > > recently, PR #4390 [1] renamed runtime kind "java" to "java:8". While a > change like this looks harmless at the first sight, it breaks all existing > actions of this kind. This may not be important for developers and > "occasional" usage of Openwhisk - but it affects production deployments. > Production deployments with existing actions require additional migration > steps when renaming or removing runtime kinds. > > I opened PR #4627 to improve documentation. Said PR also adds > "documentation" to the pre-defined Openwhisk runtime manifest files to > make developers aware that renaming or removing runtime kinds can cause > problems. > > There is another area that should be improved - but I need help to better > understand this area... > > > When trying to create an action with a kind that does not exist, a > reasonable error message is created: > > $ wsk action create kind-does-not-exist tests/dat/actions/hello.js --kind > nodejs:4 > error: Unable to create action 'kind-does-not-exist': The request content > was malformed: > kind 'nodejs:4' not in Set(dotnet:2.2, go:1.11, nodejs:10, > ballerina:0.990, ruby:2.5, nodejs:18, blackbox, swift:4.2, java:8, > sequence, nodejs:6, nodejs:12, python:3, python:2, php:7.3) (code > 33bfb55ce44d1dd0bc6e662c49ea9391) > > > When trying to display an action's metadata which has a kind that does not > exist, the resulting error message is not helpful at all: > > $ wsk action get kind-does-not-exist > error: Unable to get action 'kind-does-not-exist': Resource by this name > exists but is not in this collection. (code > 4761468230c344417fd61cdca5922e52) > > > * My conclusion from looking into controller log's and code is that > deserialization of the ExecMetaDataBase object fails with a > DeserializationException [3]. > * This exception fails the "try" block in StoreUtils.deserialize() leading > to a fall-back read in the "catch" block. This fall-back read seems to > return a WhiskTrigger instead of a WhiskActionMetaData so that a > DocumentTypeMismatchException is thrown [4]. > The resulting message can be found in controller logs: "document type > class org.apache.openwhisk.core.entity.WhiskTrigger did not match expected > type class org.apache.openwhisk.core.entity.WhiskActionMetaData.". > * As a result, getEntity() fails with the misleading error message > mentioned above and HTTP status code 409 (Conflict). > > Can somebody explain why [4] has a fall-back and which scenarios are > addressed by this? > > In our scenario, ExecMetaDataBase should probably throw an > UnknownRuntimeKindException and StoreUtils.deserialize() should not have a > fall-back for this exception. > > > [1] https://github.com/apache/openwhisk/pull/4390 > [2] https://github.com/apache/openwhisk/pull/4627 > [3] > > https://github.com/apache/openwhisk/blob/2036548e62dbf959d91c2328e86318bd7cfa656f/common/scala/src/main/scala/org/apache/openwhisk/core/entity/Exec.scala#L445-L450 > [4] > > https://github.com/apache/openwhisk/blob/2036548e62dbf959d91c2328e86318bd7cfa656f/common/scala/src/main/scala/org/apache/openwhisk/core/database/StoreUtils.scala#L58-L67 > [5] > > https://github.com/apache/openwhisk/blob/be1e3a19c02956c9be85023a0bb0ff399c21444d/core/controller/src/main/scala/org/apache/openwhisk/core/controller/ApiUtils.scala#L148-L150 > > > Mit freundlichen Grüßen / Regards, > > Sven Lange-Last > Senior Software Engineer > IBM Cloud Functions > Apache OpenWhisk > > > E-mail: sven.lange-l...@de.ibm.com > Find me on: > > > Schoenaicher Str. 220 > Boeblingen, 71032 > Germany > > > > > IBM Deutschland Research & Development GmbH > Vorsitzende des Aufsichtsrats: Martina Koederitz > Geschäftsführung: Dirk Wittkopp > Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, > HRB 243294 > > >
Dangers of renaming and removing runtime kinds
Hello Openwhisk community members, recently, PR #4390 [1] renamed runtime kind "java" to "java:8". While a change like this looks harmless at the first sight, it breaks all existing actions of this kind. This may not be important for developers and "occasional" usage of Openwhisk - but it affects production deployments. Production deployments with existing actions require additional migration steps when renaming or removing runtime kinds. I opened PR #4627 to improve documentation. Said PR also adds "documentation" to the pre-defined Openwhisk runtime manifest files to make developers aware that renaming or removing runtime kinds can cause problems. There is another area that should be improved - but I need help to better understand this area... When trying to create an action with a kind that does not exist, a reasonable error message is created: $ wsk action create kind-does-not-exist tests/dat/actions/hello.js --kind nodejs:4 error: Unable to create action 'kind-does-not-exist': The request content was malformed: kind 'nodejs:4' not in Set(dotnet:2.2, go:1.11, nodejs:10, ballerina:0.990, ruby:2.5, nodejs:18, blackbox, swift:4.2, java:8, sequence, nodejs:6, nodejs:12, python:3, python:2, php:7.3) (code 33bfb55ce44d1dd0bc6e662c49ea9391) When trying to display an action's metadata which has a kind that does not exist, the resulting error message is not helpful at all: $ wsk action get kind-does-not-exist error: Unable to get action 'kind-does-not-exist': Resource by this name exists but is not in this collection. (code 4761468230c344417fd61cdca5922e52) * My conclusion from looking into controller log's and code is that deserialization of the ExecMetaDataBase object fails with a DeserializationException [3]. * This exception fails the "try" block in StoreUtils.deserialize() leading to a fall-back read in the "catch" block. This fall-back read seems to return a WhiskTrigger instead of a WhiskActionMetaData so that a DocumentTypeMismatchException is thrown [4]. The resulting message can be found in controller logs: "document type class org.apache.openwhisk.core.entity.WhiskTrigger did not match expected type class org.apache.openwhisk.core.entity.WhiskActionMetaData.". * As a result, getEntity() fails with the misleading error message mentioned above and HTTP status code 409 (Conflict). Can somebody explain why [4] has a fall-back and which scenarios are addressed by this? In our scenario, ExecMetaDataBase should probably throw an UnknownRuntimeKindException and StoreUtils.deserialize() should not have a fall-back for this exception. [1] https://github.com/apache/openwhisk/pull/4390 [2] https://github.com/apache/openwhisk/pull/4627 [3] https://github.com/apache/openwhisk/blob/2036548e62dbf959d91c2328e86318bd7cfa656f/common/scala/src/main/scala/org/apache/openwhisk/core/entity/Exec.scala#L445-L450 [4] https://github.com/apache/openwhisk/blob/2036548e62dbf959d91c2328e86318bd7cfa656f/common/scala/src/main/scala/org/apache/openwhisk/core/database/StoreUtils.scala#L58-L67 [5] https://github.com/apache/openwhisk/blob/be1e3a19c02956c9be85023a0bb0ff399c21444d/core/controller/src/main/scala/org/apache/openwhisk/core/controller/ApiUtils.scala#L148-L150 Mit freundlichen Grüßen / Regards, Sven Lange-Last Senior Software Engineer IBM Cloud Functions Apache OpenWhisk E-mail: sven.lange-l...@de.ibm.com Find me on: Schoenaicher Str. 220 Boeblingen, 71032 Germany IBM Deutschland Research & Development GmbH Vorsitzende des Aufsichtsrats: Martina Koederitz Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
