[GitHub] dubeejw commented on issue #3206: Refactor controller role to use variables for hostname and host index
dubeejw commented on issue #3206: Refactor controller role to use variables for hostname and host index URL: https://github.com/apache/incubator-openwhisk/pull/3206#issuecomment-363335910 PG2 2755 ?? PG5 65 ?? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on issue #3206: Refactor controller role to use variables for hostname and host index
dubeejw commented on issue #3206: Refactor controller role to use variables for hostname and host index URL: https://github.com/apache/incubator-openwhisk/pull/3206#issuecomment-363335910 PG2 2754 ?? PG5 64 ?? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw opened a new pull request #3254: Do not disable rule when it is updated
dubeejw opened a new pull request #3254: Do not disable rule when it is updated URL: https://github.com/apache/incubator-openwhisk/pull/3254 Closes https://github.com/apache/incubator-openwhisk/issues/1840 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] daisy-ycguo commented on issue #17: Step 2: Verify the compliance of the source code
daisy-ycguo commented on issue #17: Step 2: Verify the compliance of the source code URL: https://github.com/apache/incubator-openwhisk-release/issues/17#issuecomment-363330861 @houshengbo I failed to run `./gradlew taredSources`. I got a NPE. But I run RAT with a folder option. `java -jar apache-rat-0.9-SNAPSHOT.jar wskdeploy_folder_name > rat_report.txt` And it creates a report. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes opened a new pull request #3253: Efficiently parse CompletionMessage.
markusthoemmes opened a new pull request #3253: Efficiently parse CompletionMessage. URL: https://github.com/apache/incubator-openwhisk/pull/3253 The parsing of `Either` relies on parsing failures in its generalized form. In this specialized case we can circumenvent this and parse only once, without throwing any forced error messages. Fixes #3228 @starpit Want to have a look? I took the liberty to open the PR. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on issue #3243: Collect test code coverage data
chetanmeh commented on issue #3243: Collect test code coverage data
URL:
https://github.com/apache/incubator-openwhisk/issues/3243#issuecomment-362995897
## How scoverage works
Scoverage [gradle plugin][1] configures a [scala compiler plugin][2] which
then instruments the source code and generates instrumented classes in
`build/classes/scala/scoverage`.
It also generates a `build/scoverage/scoverage.coverage.xml` which is a db
of source code instrumentation mapping
```xml
/absolute/path/of/openwhisk/common/scala/src/main/scala/whisk/core/entity/EntityPath.scala
whisk.core.entity
EntityPath
Object
whisk.core.entity.EntityPath
apply
/absolute/path/of/openwhisk/common/scala/src/main/scala/whisk/core/entity/EntityPath.scala
3331
5251
5309
149
scala.this.Predef.require(parts.!=(null).(parts.nonEmpty),
path undefined)
scala.Predef.require
Apply
false
0
false
...
```
The instrumented source looks like below (post decompile). Here call is
being made to `scoverage.Invoker.` with the position id
```java
public Seq whisk$core$entity$EntityPath$$apply(Seq parts)
throws IllegalArgumentException
{
Invoker..MODULE$.invoked(3331,
"/absolute/path/of/openwhisk/openwhisk/common/scala/build/scoverage");
Invoker..MODULE$.invoked(3329,
"/absolute/path/of/openwhisk/openwhisk/common/scala/build/scoverage");
Invoker..MODULE$.invoked(3327,
"/absolute/path/of/apache/openwhisk/openwhisk/common/scala/build/scoverage");
Object
```
Now as test execute and the code path gets executed it would record the
invocation in `build/scoverage/scoverage.measurements.xxx` files (xxx maps to
threadId). Post test complete a task is run which consumes the measurement file
and scoverage.coverage.xml to produce a `build/reports/scoverage/scoverage.xml`
which the coverage file for that module. Then multiple coverage file can be
combined to produce the final report
### Runtime classpath implication
For the instrumented code to work we need to include following deps in
classpath (current version 1.3.1)
1. org.scoverage:scalac-scoverage-plugin_2.11
2. org.scoverage:scalac-scoverage-runtime_2.11
This is simpler for unit test or tests where code from common, invoker and
controller is run in same vm as test cases as the scoverage gradle plugin
adapts the classpaths. However it posses some challenge with Docker setups
[1]: https://plugins.gradle.org/plugin/org.scoverage
[2]: https://github.com/scoverage/scalac-scoverage-plugin
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] chetanmeh commented on issue #3243: Collect test code coverage data
chetanmeh commented on issue #3243: Collect test code coverage data URL: https://github.com/apache/incubator-openwhisk/issues/3243#issuecomment-363160950 ### Code coverage and Docker execution For capturing coverage stats for tests which hit the controller and invoker docker containers we would need following changes 1. Include the scoverage jars in the docker container and launch script classpath 2. Include the instrumented jars for controller and invoker and ensure that they come before in classpath 3. Provide a writable mount to `/core/controller/build/scoverage` directory (+ for invoker) such that measurement files can be written. The volume mapping should point to different directory on filesystem as scoverage uses thread id for making measurement file. As the measurement is being written from threads in different vm the ids may collide. This can possibly be done by producing 2 extra containers which extend the main containers and have been customized for coverage. Further the ansible scripts would need to pick these containers This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #3252: bump docker runtime for large input, perl
csantanapr commented on issue #3252: bump docker runtime for large input, perl URL: https://github.com/apache/incubator-openwhisk/pull/3252#issuecomment-363317240 Doc updates would be cover by different PR. I want to do refactor the whole native/docker section. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #3252: bump docker runtime for large input, perl
csantanapr commented on issue #3252: bump docker runtime for large input, perl URL: https://github.com/apache/incubator-openwhisk/pull/3252#issuecomment-363316876 PG 3 1800 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr opened a new pull request #3252: bump docker runtime for large input, perl
csantanapr opened a new pull request #3252: bump docker runtime for large input, perl URL: https://github.com/apache/incubator-openwhisk/pull/3252 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw opened a new pull request #3251: Refactor List Limit Error Message
dubeejw opened a new pull request #3251: Refactor List Limit Error Message URL: https://github.com/apache/incubator-openwhisk/pull/3251 The current error message does not make since if the limit provided is less than zero. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on issue #3170: Add controller info route test
dubeejw commented on issue #3170: Add controller info route test URL: https://github.com/apache/incubator-openwhisk/pull/3170#issuecomment-363309624 @rabbah, moved the test to another package. Anything else? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on issue #3170: Add controller info route test
dubeejw commented on issue #3170: Add controller info route test URL: https://github.com/apache/incubator-openwhisk/pull/3170#issuecomment-363309624 @rabbah, moved the test to another package? Anything else? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3206: Refactor controller role to use variables for hostname and host index
dubeejw commented on a change in pull request #3206: Refactor controller role
to use variables for hostname and host index
URL:
https://github.com/apache/incubator-openwhisk/pull/3206#discussion_r166184231
##
File path: ansible/roles/controller/tasks/deploy.yml
##
@@ -48,17 +56,17 @@
- name: prepare controller ports
set_fact:
-ports_to_expose: ["{{ controller.basePort +
groups['controllers'].index(inventory_hostname) }}:8080", "{{
controller.akka.cluster.basePort +
groups['controllers'].index(inventory_hostname) }}:{{
controller.akka.cluster.bindPort }}"]
+ports_to_expose: ["{{ controller.basePort + (controller_index | int)
}}:8080", "{{ controller.akka.cluster.basePort + (controller_index | int) }}:{{
controller.akka.cluster.bindPort }}"]
Review comment:
@cbickel, unfortunately defining the `controller_index` as an `int` results
in the variable type being `unicode`. See below.
Define `controller_index` as an int:
```
- name: get controller index
set_fact:
controller_index: "{{ groups['controllers'].index(inventory_hostname) |
int }}"
```
Attempting to add `controller_index` to an int:
```
{{ controller.basePort + controller_index }}
```
Results in this error:
```
fatal: [192.168.99.100]: FAILED! => {"failed": true, "msg": "Unexpected
templating type error occurred on ({{ controller.basePort + controller_index
}}:8080): unsupported operand type(s) for +: 'int' and 'unicode'"}
```
Apparently using `set_fact` always produces a string:
https://github.com/ansible/ansible/issues/4170.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166171592
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesContainer.scala
##
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.time.Instant
+import java.util.concurrent.atomic.AtomicReference
+
+import akka.stream.StreamLimitReachedException
+import akka.stream.scaladsl.Framing.FramingException
+import akka.stream.scaladsl.{Framing, Source}
+import akka.util.ByteString
+
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import spray.json._
+import whisk.common.Logging
+import whisk.common.TransactionId
+import whisk.core.containerpool.Container
+import whisk.core.containerpool.WhiskContainerStartupError
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.{CompleteAfterOccurrences,
DockerContainer, OccurrencesNotFoundException}
+import whisk.core.containerpool.logging.LogLine
+import whisk.core.entity.ByteSize
+import whisk.http.Messages
+
+object KubernetesContainer {
+
+ /**
+ * Creates a container running in kubernetes
+ *
+ * @param transid transaction creating the container
+ * @param image image to create the container from
+ * @param userProvidedImage whether the image is provided by the user
+ * or is an OpenWhisk provided image
+ * @param labels labels to set on the container
+ * @param name optional name for the container
+ * @return a Future which either completes with a KubernetesContainer or one
of two specific failures
+ */
+ def create(transid: TransactionId,
+ image: String,
+ userProvidedImage: Boolean = false,
+ environment: Map[String, String] = Map(),
+ labels: Map[String, String] = Map(),
+ name: Option[String] = None)(implicit kubernetes: KubernetesApi,
+ ec: ExecutionContext,
+ log: Logging):
Future[KubernetesContainer] = {
+implicit val tid = transid
+
+val podName = name.getOrElse("").replace("_", "-").replaceAll("[()]",
"").toLowerCase()
+for {
+ id <- kubernetes.run(image, podName, environment, labels).recoverWith {
+case _ => Future.failed(WhiskContainerStartupError(s"Failed to run
container with image '${image}'."))
+ }
+ ip <- kubernetes.inspectIPAddress(id).recoverWith {
+// remove the container immediately if inspect failed as
+// we cannot recover that case automatically
+case _ =>
+ kubernetes.rm(id)
+ Future.failed(WhiskContainerStartupError(s"Failed to obtain IP
address of container '${id.asString}'."))
+ }
+} yield new KubernetesContainer(id, ip)
+ }
+}
+
+/**
+ * Represents a container as run by kubernetes.
+ *
+ * This class contains OpenWhisk specific behavior and as such does not
necessarily
+ * use kubernetes commands to achieve the effects needed.
+ *
+ * @constructor
+ * @param id the id of the container
+ * @param addr the ip & port of the container
+ */
+class KubernetesContainer(protected val id: ContainerId, protected val addr:
ContainerAddress)(
+ implicit kubernetes: KubernetesApi,
+ protected val ec: ExecutionContext,
+ protected val logging: Logging)
+extends Container {
+
+ /** The last read timestamp in the log file */
+ private val lastTimestamp = new AtomicReference[Option[String]](None)
+
+ // no-op under Kubernetes
+ def suspend()(implicit transid: TransactionId): Future[Unit] =
Future.successful({})
+
+ // no-op under Kubernetes
+ def resume()(implicit transid: TransactionId): Future[Unit] =
Future.successful({})
+
+ override def destroy()(implicit transid: TransactionId): Future[Unit] = {
+super.destroy()
+kubernetes.rm(id)
+ }
+
+ def logs(limit: ByteSize, waitForSentinel: Boolean)(implicit transid:
TransactionId): Source[ByteString, Any] = {
+
+val activationMarkerCheck:
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166171460
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
[GitHub] csantanapr commented on issue #3250: bump nodejs openwhisk npm to 3.12.0
csantanapr commented on issue #3250: bump nodejs openwhisk npm to 3.12.0 URL: https://github.com/apache/incubator-openwhisk/pull/3250#issuecomment-363283066 PG 4 1316 looks OK This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ScottChapman commented on issue #704: Unable to specify Raw HTTP
ScottChapman commented on issue #704: Unable to specify Raw HTTP URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/704#issuecomment-363279321 how often are new builds produced? Ones in the releases folder seem kind old(ish) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166162686
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166162501
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166160272
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] bwmcadams commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
bwmcadams commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166159398
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
+
[GitHub] dgrove-oss commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
dgrove-oss commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166135470
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,222 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.net.URL
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.ErrorLevel
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import spray.json._
+import spray.json.DefaultJsonProtocol._
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import io.fabric8.kubernetes.client.utils.URLUtils
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+"whisk.kubernetes.timeouts"))(executionContext: ExecutionContext)(implicit
log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map { _ =>
+name
+ }
+ .map(ContainerId.apply)
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
Review comment:
Circling back on the client question. The fabric8io client appears to lack
support for specifying pod affinity. Skimming through the source code, I see
no support for affinity (and several open issues on GitHub asking for affinity
support). The
[GitHub] mrutkows closed pull request #707: Adding support for raw HTTP web actions
mrutkows closed pull request #707: Adding support for raw HTTP web actions
URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/707
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/parsers/manifest_parser.go b/parsers/manifest_parser.go
index d913c1c8..c5eb0c80 100644
--- a/parsers/manifest_parser.go
+++ b/parsers/manifest_parser.go
@@ -652,9 +652,12 @@ func (dm *YAMLParser) ComposeActions(filePath string,
actions map[string]Action,
/*
* Web Export
*/
- // TODO() add boolean value const
- if action.Webexport == "true" {
- wskaction.Annotations, errorParser =
utils.WebAction("yes", listOfAnnotations, false)
+ // Treat ACTION as a web action, a raw HTTP web action, or as a
standard action based on web-export;
+ // when web-export is set to yes | true, treat action as a web
action,
+ // when web-export is set to raw, treat action as a raw HTTP
web action,
+ // when web-export is set to no | false, treat action as a
standard action
+ if len(action.Webexport) != 0 {
+ wskaction.Annotations, errorParser =
utils.WebAction(filePath, action.Name, action.Webexport, listOfAnnotations,
false)
if errorParser != nil {
return s1, errorParser
}
diff --git a/parsers/manifest_parser_test.go b/parsers/manifest_parser_test.go
index 5b45dc16..ae640338 100644
--- a/parsers/manifest_parser_test.go
+++ b/parsers/manifest_parser_test.go
@@ -949,9 +949,21 @@ func TestComposeActionsForWebActions(t *testing.T) {
`package:
name: helloworld
actions:
-hello:
+hello1:
+ function: ../tests/src/integration/helloworld/actions/hello.js
+ web-export: true
+hello2:
function: ../tests/src/integration/helloworld/actions/hello.js
- web-export: true`
+ web-export: yes
+hello3:
+ function: ../tests/src/integration/helloworld/actions/hello.js
+ web-export: raw
+hello4:
+ function: ../tests/src/integration/helloworld/actions/hello.js
+ web-export: false
+hello5:
+ function: ../tests/src/integration/helloworld/actions/hello.js
+ web-export: no`
dir, _ := os.Getwd()
tmpfile, err := ioutil.TempFile(dir,
"manifest_parser_validate_web_actions_")
if err == nil {
@@ -963,7 +975,7 @@ func TestComposeActionsForWebActions(t *testing.T) {
actions, err := p.ComposeActionsFromAllPackages(m,
tmpfile.Name(), whisk.KeyValue{})
if err == nil {
for i := 0; i < len(actions); i++ {
- if actions[i].Action.Name == "hello" {
+ if actions[i].Action.Name == "hello1" {
for _, a := range
actions[i].Action.Annotations {
switch a.Key {
case "web-export":
@@ -974,6 +986,50 @@ func TestComposeActionsForWebActions(t *testing.T) {
assert.Equal(t,
true, a.Value, "Expected true for final but got
"+strconv.FormatBool(a.Value.(bool)))
}
}
+ } else if actions[i].Action.Name ==
"hello2" {
+ for _, a := range
actions[i].Action.Annotations {
+ switch a.Key {
+ case "web-export":
+ assert.Equal(t,
true, a.Value, "Expected true for web-export but got
"+strconv.FormatBool(a.Value.(bool)))
+ case "raw-http":
+ assert.Equal(t,
false, a.Value, "Expected false for raw-http but got
"+strconv.FormatBool(a.Value.(bool)))
+ case "final":
+ assert.Equal(t,
true, a.Value, "Expected true for final but got
"+strconv.FormatBool(a.Value.(bool)))
+ }
+ }
+ } else if actions[i].Action.Name ==
"hello3" {
+ for _, a
[GitHub] mrutkows closed issue #704: Unable to specify Raw HTTP
mrutkows closed issue #704: Unable to specify Raw HTTP URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/704 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo opened a new pull request #219: Add the support to package source code with Gradle
houshengbo opened a new pull request #219: Add the support to package source code with Gradle URL: https://github.com/apache/incubator-openwhisk-cli/pull/219 The task taredSources is added to generate the source code package. The task cleanBuild is added to remove all the files under the folder build. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #20: Step 6: Package the artifacts
houshengbo commented on issue #20: Step 6: Package the artifacts URL: https://github.com/apache/incubator-openwhisk-release/issues/20#issuecomment-363239349 Let's start from packaging source code. All the openwhisk repos to be released need to have a gradlew task named "taredSources" to package the source code under build folder, and cleanBuild to remove all the files under build folder in each repo. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #17: Step 2: Verify the compliance of the source code
houshengbo commented on issue #17: Step 2: Verify the compliance of the source code URL: https://github.com/apache/incubator-openwhisk-release/issues/17#issuecomment-363233660 @daisy-ycguo This is is related to https://github.com/apache/incubator-openwhisk-release/issues/5 I have created a gradle support to package the source code of wskdeploy, which is merged already: https://github.com/apache/incubator-openwhisk-wskdeploy You can have the source code package generated under build folder of incubator-openwhisk-wskdeploy, by running ./gradlew taredSources. You can use the wskdeploy tar as example. Please try to see if you can develop a bash script, to verify all the sources in a tar.gz file contain the apache headers, by using RAT. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vipulkashyap111 opened a new pull request #155: #112 Adding feature to insert document when overwrite and putIfAbsent?
vipulkashyap111 opened a new pull request #155: #112 Adding feature to insert document when overwrite and putIfAbsent? URL: https://github.com/apache/incubator-openwhisk-package-cloudant/pull/155 ? true This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #17: Step 2: Verify the compliance of the source code
houshengbo commented on issue #17: Step 2: Verify the compliance of the source code URL: https://github.com/apache/incubator-openwhisk-release/issues/17#issuecomment-363233660 @daisy-ycguo This is is related to https://github.com/apache/incubator-openwhisk-release/issues/5 I have created a gradle support to package the source code of wskdeploy, which is merged already: https://github.com/apache/incubator-openwhisk-wskdeploy You can have the source code package generated under build folder of incubator-openwhisk-wskdeploy, by running ./gradlew taredSources. Please try to see if you can develop a bash script, to verify all the sources in a tar.gz file contain the apache headers, by using RAT. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #17: Step 2: Verify the compliance of the source code
houshengbo commented on issue #17: Step 2: Verify the compliance of the source code URL: https://github.com/apache/incubator-openwhisk-release/issues/17#issuecomment-363233660 @daisy-ycguo This is is related to https://github.com/apache/incubator-openwhisk-release/issues/5 I have created a gradle support to package the source code of wskdeploy, which is merged already: https://github.com/apache/incubator-openwhisk-wskdeploy Please try to see if you can develop a bash script, to verify all the sources in a tar.gz file contain the apache headers, by using RAT. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #17: Step 2: Verify the compliance of the source code
houshengbo commented on issue #17: Step 2: Verify the compliance of the source code URL: https://github.com/apache/incubator-openwhisk-release/issues/17#issuecomment-363233660 This is is related to https://github.com/apache/incubator-openwhisk-release/issues/5 I have created a gradle support to package the source code of wskdeploy, which is merged already: https://github.com/apache/incubator-openwhisk-wskdeploy Please try to see if you can develop a bash script, to verify all the sources in a tar.gz file contain the apache headers, by using RAT. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #17: Step 2: Verify the compliance of the source code
houshengbo commented on issue #17: Step 2: Verify the compliance of the source code URL: https://github.com/apache/incubator-openwhisk-release/issues/17#issuecomment-363233660 issue 5 I have created a gradle support to package the source code of wskdeploy, which is merged already: https://github.com/apache/incubator-openwhisk-wskdeploy Please try to see if you can develop a bash script, to verify all the sources in a tar.gz file contain the apache headers. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3187: reduce rule activation records
rabbah commented on issue #3187: reduce rule activation records URL: https://github.com/apache/incubator-openwhisk/pull/3187#issuecomment-363232706 Yes but if you rebased to master already and it?s still there then something else is off. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3187: reduce rule activation records
rabbah commented on issue #3187: reduce rule activation records URL: https://github.com/apache/incubator-openwhisk/pull/3187#issuecomment-363232706 I?d think so but if you rebased to master already and it?s still there then something else is off. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mdeuser commented on issue #3187: reduce rule activation records
mdeuser commented on issue #3187: reduce rule activation records URL: https://github.com/apache/incubator-openwhisk/pull/3187#issuecomment-363228820 can i assume that since the `reduce fillInStackTrace expenses` commit is already in the master repo, that this same commit being in this PR will be ignored during merge? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mdeuser commented on a change in pull request #3187: reduce rule activation records
mdeuser commented on a change in pull request #3187: reduce rule activation
records
URL:
https://github.com/apache/incubator-openwhisk/pull/3187#discussion_r166118176
##
File path: core/controller/src/main/scala/whisk/core/controller/Triggers.scala
##
@@ -413,12 +413,19 @@ trait WhiskTriggersApi extends WhiskCollectionAPI {
actionActivationId: Option[JsValue] = None,
errorMsg: Option[String] = None): JsObject = {
JsObject(
- "rule" -> JsString(ruleName.asString),
- "action" -> JsString(actionName.asString),
- "statusCode" -> JsNumber(statusCode),
- "success" -> JsBoolean(statusCode == ActivationResponse.Success),
- "activationId" -> actionActivationId.toJson,
- "error" -> errorMsg.toJson)
+ Map(
+"rule" -> JsString(ruleName.asString),
+"action" -> JsString(actionName.asString),
+"statusCode" -> JsNumber(statusCode),
+"success" -> JsBoolean(statusCode == ActivationResponse.Success)) ++ {
+actionActivationId map { id =>
Review comment:
so many ways in scala to `map` :-)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] mrutkows closed pull request #706: Add the support to package the source code
mrutkows closed pull request #706: Add the support to package the source code
URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/706
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build.gradle b/build.gradle
index 1cf4798b..965aa968 100644
--- a/build.gradle
+++ b/build.gradle
@@ -20,6 +20,41 @@ ext.dockerContainerName = "wskdeploy"
ext.dockerBuildArgs = getDockerBuildArgs()
apply from: 'gradle/docker.gradle'
+project.ext {
+basePackageName = "openwhisk-wskdeploy"
+packageExtension = "tar.gz"
+if (project.hasProperty('projectVersion')) {
+packageVersion = "${projectVersion}"
+} else {
+packageVersion = ""
+}
+buildFolder = "build"
+}
+
+task taredSources(type: Tar) {
+baseName basePackageName
+description "Creates a combined tar.gz file of wskdeploy's sources"
+group "Release artifact"
+classifier "sources"
+
+from(project.rootDir) {
+include('cmd/*.go', 'deployers/*.go', 'parsers/*.go', 'utils/*.go',
+'wskderrors/*.go', 'wskenv/*.go', 'wskprint/*.go',
'wski18n/**')
+include('*.go')
+include('gradle/**')
+include('README.md', 'CONTRIBUTING.md', 'DEPENDENCIES.md')
+include('gradlew', 'gradlew.bat', 'Dockerfile', 'build.gradle')
+include('LICENSE', 'NOTICE', 'CHANGELOG')
+}
+destinationDir file(buildFolder)
+extension packageExtension
+version packageVersion
+compression = Compression.GZIP
+}
+
+task cleanBuild(type: Delete) {
+delete file(buildFolder).listFiles()
+}
task removeBinary(type: Delete) {
delete "${projectDir}/bin/wskdeploy"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166111823
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] bbrowning commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
bbrowning commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166092584
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
Review comment:
This may be specific to OpenShift, but the reason for this is so that these
pods don't get treated as temporary pods with different lifecycle semantics,
sometimes resulting in them being prematurely killed. By setting `restart` to
`Always` we prevent them from being treated as temporary pods. However, if you
only set that, then instead of creating a Pod object it ends up creating a
replication
[GitHub] kpavel commented on issue #681: proper handling of 'default' package name
kpavel commented on issue #681: proper handling of 'default' package name URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/681#issuecomment-363195995 Agree with @lionelvillard, rules and triggers should be placed at the top level to reflect OW model. Maybe should consider to place sequences there as well. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr opened a new pull request #3250: bump nodejs openwhisk npm to 3.12.0
csantanapr opened a new pull request #3250: bump nodejs openwhisk npm to 3.12.0 URL: https://github.com/apache/incubator-openwhisk/pull/3250 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mhamann closed pull request #272: OpenRESTy version bump + Power
mhamann closed pull request #272: OpenRESTy version bump + Power
URL: https://github.com/apache/incubator-openwhisk-apigateway/pull/272
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/Dockerfile b/Dockerfile
index 24c80f5..918bf3f 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -15,7 +15,7 @@ RUN apk --update add \
&& rm -rf /var/cache/apk/*
# openresty build
-ENV OPENRESTY_VERSION=1.9.7.3 \
+ENV OPENRESTY_VERSION=1.13.6.1 \
NAXSI_VERSION=0.53-2 \
PCRE_VERSION=8.37 \
TEST_NGINX_VERSION=0.24 \
@@ -46,6 +46,17 @@ RUN if [ x`uname -m` = xs390x ]; then \
&& rm -rf /tmp/luajit \
; fi
+RUN if [ x`uname -m` = xppc64le ]; then \
+ echo "Building LuaJIT for ppc64le" \
+ && mkdir /tmp/luajit \
+ && cd /tmp/luajit \
+ && curl -k -L https://api.github.com/repos/PPC64/LuaJIT/tarball >
luajit.tar.gz \
+ && tar -zxf luajit.tar.gz \
+ && cd PPC64-LuaJIT-* \
+ && make && make install \
+ && rm -rf /tmp/luajit \
+ ; fi
+
RUN echo " ... adding Openresty, NGINX, NAXSI and PCRE" \
&& mkdir -p /tmp/api-gateway \
&& readonly NPROC=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || 1) \
@@ -63,6 +74,9 @@ RUN echo " ... adding Openresty, NGINX, NAXSI and PCRE" \
&& if [ x`uname -m` = xs390x ]; then \
luajitdir="=/usr/local/" \
pcrejit="" \
+; elif [ x`uname -m` = xppc64le ]; then \
+ luajitdir="=/usr/local/" \
+ pcrejit="--with-pcre-jit" \
; else \
luajitdir="" \
pcrejit="--with-pcre-jit" \
@@ -165,7 +179,7 @@ RUN echo " ... installing opm..." \
&& cd ${_prefix}/api-gateway \
&& mkdir -p site/manifest site/pod \
&& cd site \
-&& ln -s ../lualib ./ \
+&& ( [ -x ../lualib ] || ln -s ../lualib ./ ) \
&& ln -s ${_prefix}/api-gateway/bin/opm /usr/bin/opm \
&& ln -s ${_prefix}/api-gateway/bin/resty /usr/bin/resty \
&& rm -rf /tmp/api-gateway
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rabbah commented on issue #3187: reduce rule activation records
rabbah commented on issue #3187: reduce rule activation records URL: https://github.com/apache/incubator-openwhisk/pull/3187#issuecomment-363187703 rebase to drop second commit? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #3187: reduce rule activation records
rabbah commented on a change in pull request #3187: reduce rule activation
records
URL:
https://github.com/apache/incubator-openwhisk/pull/3187#discussion_r166077958
##
File path: core/controller/src/main/scala/whisk/core/controller/Triggers.scala
##
@@ -413,12 +413,19 @@ trait WhiskTriggersApi extends WhiskCollectionAPI {
actionActivationId: Option[JsValue] = None,
errorMsg: Option[String] = None): JsObject = {
JsObject(
- "rule" -> JsString(ruleName.asString),
- "action" -> JsString(actionName.asString),
- "statusCode" -> JsNumber(statusCode),
- "success" -> JsBoolean(statusCode == ActivationResponse.Success),
- "activationId" -> actionActivationId.toJson,
- "error" -> errorMsg.toJson)
+ Map(
+"rule" -> JsString(ruleName.asString),
+"action" -> JsString(actionName.asString),
+"statusCode" -> JsNumber(statusCode),
+"success" -> JsBoolean(statusCode == ActivationResponse.Success)) ++ {
+actionActivationId map { id =>
Review comment:
you can simplify this to:
```scala
JsObject(
Map(
"rule" -> JsString(ruleName.asString),
"action" -> JsString(actionName.asString),
"statusCode" -> JsNumber(statusCode),
"success" -> JsBoolean(statusCode == ActivationResponse.Success)) ++
actionActivationId.map("activationId" -> _.toJson) ++
errorMsg.map("error" -> JsString(_)))
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] jcrossley3 commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
jcrossley3 commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166076063
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
+
[GitHub] jcrossley3 commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
jcrossley3 commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166073411
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
Review comment:
#1 makes sense. I'm gonna refer to @bbrowning for #2 and #3
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache
[GitHub] jcrossley3 commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
jcrossley3 commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166072862
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
+
[GitHub] jcrossley3 commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
jcrossley3 commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166072639
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
+
[GitHub] rabbah commented on issue #3167: Remove API for listing all entities in namespace
rabbah commented on issue #3167: Remove API for listing all entities in namespace URL: https://github.com/apache/incubator-openwhisk/pull/3167#issuecomment-363181725 @csantanapr The node runtimes have been updated, and the CLI as well. I think this is ready then. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr closed pull request #20: Bump npm openwhisk version to pick up latest release.
csantanapr closed pull request #20: Bump npm openwhisk version to pick up
latest release.
URL: https://github.com/apache/incubator-openwhisk-runtime-nodejs/pull/20
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core/nodejs6Action/CHANGELOG.md b/core/nodejs6Action/CHANGELOG.md
index 6b16920..a3c7006 100644
--- a/core/nodejs6Action/CHANGELOG.md
+++ b/core/nodejs6Action/CHANGELOG.md
@@ -1,6 +1,9 @@
# NodeJS 6 OpenWhisk Runtime Container
+## 1.5.0
+Change: Update npm openwhisk package
+- [openwhisk v3.12.0](https://www.npmjs.com/package/openwhisk) - JavaScript
client library for the OpenWhisk platform. Provides a wrapper around the
OpenWhisk APIs.
## 1.4.0
Change: Update nodejs and openwhisk npm package
diff --git a/core/nodejs6Action/Dockerfile b/core/nodejs6Action/Dockerfile
index 02b403c..102af6d 100644
--- a/core/nodejs6Action/Dockerfile
+++ b/core/nodejs6Action/Dockerfile
@@ -46,7 +46,7 @@ nano@6.2.0 \
node-uuid@1.4.7 \
nodemailer@2.6.4 \
oauth2-server@2.4.1 \
-openwhisk@3.11.0 \
+openwhisk@3.12.0 \
pkgcloud@1.4.0 \
process@0.11.9 \
pug@">=2.0.0-beta6 <2.0.1" \
diff --git a/core/nodejs8Action/CHANGELOG.md b/core/nodejs8Action/CHANGELOG.md
index 51259e9..3c93b9d 100644
--- a/core/nodejs8Action/CHANGELOG.md
+++ b/core/nodejs8Action/CHANGELOG.md
@@ -1,5 +1,10 @@
# NodeJS 8 OpenWhisk Runtime Container
+## 1.2.0
+Change: Update npm openwhisk package
+
+- [openwhisk v3.12.0](https://www.npmjs.com/package/openwhisk) - JavaScript
client library for the OpenWhisk platform. Provides a wrapper around the
OpenWhisk APIs.
+
## 1.1.0
Change: Update nodejs and openwhisk npm package
diff --git a/core/nodejs8Action/package.json b/core/nodejs8Action/package.json
index f331190..43a8e08 100644
--- a/core/nodejs8Action/package.json
+++ b/core/nodejs8Action/package.json
@@ -8,7 +8,7 @@
},
"license": "Apache-2.0",
"dependencies": {
-"openwhisk": "3.11.0",
+"openwhisk": "3.12.0",
"body-parser": "1.18.2",
"express": "4.16.2"
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding.
markusthoemmes commented on a change in pull request #3240: Add a loadbalancer
with local state and horizontal invoker sharding.
URL:
https://github.com/apache/incubator-openwhisk/pull/3240#discussion_r166059361
##
File path:
core/controller/src/main/scala/whisk/core/loadBalancer/ShardingContainerPoolBalancer.scala
##
@@ -0,0 +1,439 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.loadBalancer
+
+import java.nio.charset.StandardCharsets
+import java.util.concurrent.atomic.LongAdder
+import java.util.concurrent.ThreadLocalRandom
+
+import akka.actor.{Actor, ActorSystem, Props}
+import akka.cluster.ClusterEvent._
+import akka.cluster.{Cluster, Member, MemberStatus}
+import akka.event.Logging.InfoLevel
+import akka.stream.ActorMaterializer
+import org.apache.kafka.clients.producer.RecordMetadata
+import pureconfig._
+import whisk.common.{ForcableSemaphore, Logging, LoggingMarkers, TransactionId}
+import whisk.core.WhiskConfig._
+import whisk.core.connector._
+import whisk.core.entity._
+import whisk.core.{ConfigKeys, WhiskConfig}
+import whisk.spi.SpiLoader
+
+import scala.annotation.tailrec
+import scala.collection.concurrent.TrieMap
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContext, Future, Promise}
+import scala.util.{Failure, Success}
+
+/**
+ * A loadbalancer that uses "horizontal" sharding to not collide with fellow
loadbalancers.
+ *
+ * Horizontal sharding means, that each invoker's capacity is evenly divided
between the loadbalancers. If an invoker
+ * has at most 16 slots available, those will be divided to 8 slots for each
loadbalancer (if there are 2).
+ */
+class ShardingContainerPoolBalancer(config: WhiskConfig, controllerInstance:
InstanceId)(
+ implicit val actorSystem: ActorSystem,
+ logging: Logging,
+ materializer: ActorMaterializer)
+extends LoadBalancer {
+
+ private implicit val executionContext: ExecutionContext =
actorSystem.dispatcher
+
+ /** Build a cluster of all loadbalancers */
+ val seedNodesProvider = new
StaticSeedNodesProvider(config.controllerSeedNodes, actorSystem.name)
+ val cluster = Cluster(actorSystem)
+ cluster.joinSeedNodes(seedNodesProvider.getSeedNodes())
+
+ /** Used to manage an action for testing invoker health */
+ private val entityStore = WhiskEntityStore.datastore(config)
+
+ /** State related to invocations and throttling */
+ private val activations = TrieMap[ActivationId, ActivationEntry]()
+ private val activationsPerNamespace = TrieMap[UUID, LongAdder]()
+ private val totalActivations = new LongAdder()
+
+ /** State needed for scheduling. */
+ private val schedulingState = ShardingContainerPoolBalancerState()()
+
+ /**
+ * Monitors invoker supervision and the cluster to update the state
sequentially
+ *
+ * All state updates should go through this actor to guarantee, that
`updateState` and `updateCluster` are called
+ * mutually exclusive and not concurrently.
+ */
+ private val monitor = actorSystem.actorOf(Props(new Actor {
+override def preStart(): Unit = {
+ cluster.subscribe(self, classOf[MemberEvent], classOf[ReachabilityEvent])
+}
+
+// all members of the cluster that are available
+var availableMembers = Set.empty[Member]
+
+override def receive: Receive = {
+ case CurrentInvokerPoolState(newState) =>
+schedulingState.updateInvokers(newState)
+
+ // State of the cluster as it is right now
+ case CurrentClusterState(members, _, _, _, _) =>
+availableMembers = members.filter(_.status == MemberStatus.Up)
+schedulingState.updateCluster(availableMembers.size)
+
+ // General lifecycle events and events concerning the reachability of
members. Split-brain is not a huge concern
+ // in this case as only the invoker-threshold is adjusted according to
the perceived cluster-size.
+ // Taking the unreachable member out of the cluster from that
point-of-view results in a better experience
+ // even under split-brain-conditions, as that (in the worst-case)
results in premature overloading of invokers vs.
+ // going into overflow mode prematurely.
+ case event: ClusterDomainEvent =>
+
[GitHub] jonpspri commented on a change in pull request #3216: Download Nginx Tarball from OpenWhisk CLI release, rather than constructing file-by-file
jonpspri commented on a change in pull request #3216: Download Nginx Tarball
from OpenWhisk CLI release, rather than constructing file-by-file
URL:
https://github.com/apache/incubator-openwhisk/pull/3216#discussion_r166058016
##
File path: ansible/roles/cli/tasks/deploy.yml
##
@@ -1,29 +1,81 @@
---
# Tasks for handling CLI customization and publishing
+#
+# Note: The configuration directory is actually located on the local machine;
+#this script is run under the local host, usually 172.17.0.1 (docker
local)
-- name: "ensure nginx directory for cli exists"
+- name: "Ensure nginx directory for cli exists"
file:
-path: "{{ cli.nginxdir }}"
+path: "{{ openwhisk_cli.nginxdir.name }}"
state: directory
- become: "{{ cli.dir.become }}"
+ become: "{{ openwhisk_cli.nginxdir.become }}"
-- set_fact:
-cli_installation_mode="{{ openwhisk_cli.installation_mode }}"
+- name: "Ensure OpenWhisk build directory exists (for temp archive work)"
+ file:
+path: "{{ openwhisk_build_dir }}/{{ openwhisk_cli.archive_name }}"
+state: directory
+
+#
+# Why are we unarchiving into the build directory instead of directly into
+# the Nginx config directory? Because the Nginx config directory is (by
+# default) located in the /tmp/... directory tree, which has a sticky bit
+# set. Said sticky bit creates no end of troubles for tar, so we're going
+# to just avoid it entirely, rather than muck about with who's got which tar
+# with which right tar options installed where. It makes for many more
+# items in this ansible playbook than we'd hoped, but at least it's (fairly)
+# straightforward.
+#
+- name: "Download release archive to build directory ..."
+ get_url:
+url: "{{ openwhisk_cli.remote.location }}/{{
openwhisk_cli.archive_name}}-{{ openwhisk_cli_tag }}-all.tgz"
Review comment:
Why? My entire purpose in starting down this path was to reduce complexity.
Re-storing the list-based downloads adds back in:
1. Adjustment/re-generation of content.json
2. Specification and parsing of local build architecture(s)
3. Figuring out URLs for particular archives (because of the odd
structuring rules).
3. Circular dependencies -- when an architecture is added to the CLI build
it must be added here.
For what benefit? Saving a few megabytes of disk space and trimming a
directory structure that is unseen. Users running the CLI still have the
option of downloading the appropriate architecture from the nginx server that's
running. So I suppose my core question is "who are 'users' in the conversation
above"?
I'll do the change if it's what's needed to get this off my plate, but the
whole rationale was to avoid complicating the build and to avoid having a
circular dependency where the list of architectures was to be kept in the
`incubator-openwhisk` repository. If we're doing individual downloads, I'd
abandon the archive download regardless of network connections.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166057090
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
Review comment:
1. Looks to me like `memory=256Mi` should be passed in from the outside.
2. Why `restart Always`? For user-containers?
3. Do we need to make the generator configurable?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific
[GitHub] jonpspri commented on issue #3216: Download Nginx Tarball from OpenWhisk CLI release, rather than constructing file-by-file
jonpspri commented on issue #3216: Download Nginx Tarball from OpenWhisk CLI release, rather than constructing file-by-file URL: https://github.com/apache/incubator-openwhisk/pull/3216#issuecomment-363169860 @rabbah For a locally build API, the archive is sourced from the local build. The flags to declare which approach to take remain. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166058082
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesContainerFactory.scala
##
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import akka.actor.ActorSystem
+
+import scala.concurrent.Await
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.duration._
+
+import whisk.common.Logging
+import whisk.common.TransactionId
+import whisk.core.containerpool.Container
+import whisk.core.containerpool.ContainerFactory
+import whisk.core.containerpool.ContainerFactoryProvider
+import whisk.core.entity.ByteSize
+import whisk.core.entity.ExecManifest.ImageName
+import whisk.core.entity.InstanceId
+import whisk.core.WhiskConfig
+
+class KubernetesContainerFactory(label: String, config: WhiskConfig)(implicit
actorSystem: ActorSystem,
+ ec:
ExecutionContext,
+ logging:
Logging)
+extends ContainerFactory {
+
+ implicit val kubernetes = new KubernetesClient()(ec)
+
+ /** Perform cleanup on init */
+ override def init(): Unit = cleanup()
+
+ override def cleanup() = {
+logging.info(this, "Cleaning up function runtimes")
+val cleaning = kubernetes.rm("invoker", label)(TransactionId.invokerNanny)
+Await.ready(cleaning, 30.seconds)
+ }
+
+ override def createContainer(tid: TransactionId,
+ name: String,
+ actionImage: ImageName,
+ userProvidedImage: Boolean,
+ memory: ByteSize)(implicit config: WhiskConfig,
logging: Logging): Future[Container] = {
+val image = if (userProvidedImage) {
+ actionImage.publicImageName
+} else {
+ actionImage.localImageName(config.dockerRegistry,
config.dockerImagePrefix, Some(config.dockerImageTag))
+}
+
+KubernetesContainer.create(
+ tid,
+ image = image,
+ userProvidedImage = userProvidedImage,
+ environment = Map("__OW_API_HOST" -> config.wskApiHost),
+ labels = Map("invoker" -> label),
+ name = Some(name))
Review comment:
Memory usage should be passed in here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166055516
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166053430
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166055516
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
[GitHub] chetanmeh commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats
chetanmeh commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats URL: https://github.com/apache/incubator-openwhisk/pull/3249#issuecomment-363164513 > Does this enable any means of integration/automization? I.e.: Flag a PR if it reduces coverage. Not yet. Once the base is in we can configure the incubator-openwhisk repo for codecov to do that like its done for js client This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3219: Kubernetes ContainerFactoryProvider implementation
markusthoemmes commented on a change in pull request #3219: Kubernetes
ContainerFactoryProvider implementation
URL:
https://github.com/apache/incubator-openwhisk/pull/3219#discussion_r166053430
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesClient.scala
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.kubernetes
+
+import java.io.FileNotFoundException
+import java.nio.file.Files
+import java.nio.file.Paths
+
+import akka.actor.ActorSystem
+import akka.event.Logging.{ErrorLevel, InfoLevel}
+import akka.http.scaladsl.model.Uri
+import akka.http.scaladsl.model.Uri.Path
+import akka.http.scaladsl.model.Uri.Query
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import pureconfig.loadConfigOrThrow
+import whisk.common.Logging
+import whisk.common.LoggingMarkers
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.containerpool.ContainerId
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.docker.ProcessRunner
+import whisk.core.containerpool.logging.LogLine
+import scala.concurrent.duration.Duration
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.blocking
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import scala.util.Try
+import spray.json._
+
+import io.fabric8.kubernetes.client.ConfigBuilder
+import io.fabric8.kubernetes.client.DefaultKubernetesClient
+import okhttp3.Request
+
+/**
+ * Configuration for kubernetes client command timeouts.
+ */
+case class KubernetesClientTimeoutConfig(run: Duration, rm: Duration, inspect:
Duration, logs: Duration)
+
+/**
+ * Serves as interface to the kubectl CLI tool.
+ *
+ * Be cautious with the ExecutionContext passed to this, as the
+ * calls to the CLI are blocking.
+ *
+ * You only need one instance (and you shouldn't get more).
+ */
+class KubernetesClient(
+ timeouts: KubernetesClientTimeoutConfig =
loadConfigOrThrow[KubernetesClientTimeoutConfig](
+ConfigKeys.kubernetesTimeouts))(executionContext:
ExecutionContext)(implicit log: Logging, as: ActorSystem)
+extends KubernetesApi
+with ProcessRunner {
+ implicit private val ec = executionContext
+ implicit private val kubeRestClient = new DefaultKubernetesClient(
+new ConfigBuilder()
+ .withConnectionTimeout(timeouts.logs.toMillis.toInt)
+ .withRequestTimeout(timeouts.logs.toMillis.toInt)
+ .build())
+
+ // Determines how to run kubectl. Failure to find a kubectl binary implies
+ // a failure to initialize this instance of KubernetesClient.
+ protected val kubectlCmd: Seq[String] = {
+val alternatives = List("/usr/bin/kubectl", "/usr/local/bin/kubectl")
+
+val kubectlBin = Try {
+ alternatives.find(a => Files.isExecutable(Paths.get(a))).get
+} getOrElse {
+ throw new FileNotFoundException(s"Couldn't locate kubectl binary (tried:
${alternatives.mkString(", ")}).")
+}
+
+Seq(kubectlBin)
+ }
+
+ def run(image: String, name: String, environment: Map[String, String] =
Map(), labels: Map[String, String] = Map())(
+implicit transid: TransactionId): Future[ContainerId] = {
+val environmentArgs = environment.flatMap {
+ case (key, value) => Seq("--env", s"$key=$value")
+}.toSeq
+
+val labelArgs = labels.map {
+ case (key, value) => s"$key=$value"
+} match {
+ case Seq() => Seq()
+ case pairs => Seq("-l") ++ pairs
+}
+
+val runArgs = Seq(
+ "run",
+ name,
+ "--image",
+ image,
+ "--generator",
+ "run-pod/v1",
+ "--restart",
+ "Always",
+ "--limits",
+ "memory=256Mi") ++ environmentArgs ++ labelArgs
+
+runCmd(runArgs, timeouts.run)
+ .map(_ => ContainerId(name))
+ }
+
+ def inspectIPAddress(id: ContainerId)(implicit transid: TransactionId):
Future[ContainerAddress] = {
+Future {
+ blocking {
+val pod =
+
kubeRestClient.pods().withName(id.asString).waitUntilReady(timeouts.inspect.length,
timeouts.inspect.unit)
+ContainerAddress(pod.getStatus().getPodIP())
+ }
[GitHub] vvraskin commented on a change in pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding.
vvraskin commented on a change in pull request #3240: Add a loadbalancer with
local state and horizontal invoker sharding.
URL:
https://github.com/apache/incubator-openwhisk/pull/3240#discussion_r166048402
##
File path:
core/controller/src/main/scala/whisk/core/loadBalancer/ShardingContainerPoolBalancer.scala
##
@@ -0,0 +1,439 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.loadBalancer
+
+import java.nio.charset.StandardCharsets
+import java.util.concurrent.atomic.LongAdder
+import java.util.concurrent.ThreadLocalRandom
+
+import akka.actor.{Actor, ActorSystem, Props}
+import akka.cluster.ClusterEvent._
+import akka.cluster.{Cluster, Member, MemberStatus}
+import akka.event.Logging.InfoLevel
+import akka.stream.ActorMaterializer
+import org.apache.kafka.clients.producer.RecordMetadata
+import pureconfig._
+import whisk.common.{ForcableSemaphore, Logging, LoggingMarkers, TransactionId}
+import whisk.core.WhiskConfig._
+import whisk.core.connector._
+import whisk.core.entity._
+import whisk.core.{ConfigKeys, WhiskConfig}
+import whisk.spi.SpiLoader
+
+import scala.annotation.tailrec
+import scala.collection.concurrent.TrieMap
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContext, Future, Promise}
+import scala.util.{Failure, Success}
+
+/**
+ * A loadbalancer that uses "horizontal" sharding to not collide with fellow
loadbalancers.
+ *
+ * Horizontal sharding means, that each invoker's capacity is evenly divided
between the loadbalancers. If an invoker
+ * has at most 16 slots available, those will be divided to 8 slots for each
loadbalancer (if there are 2).
+ */
+class ShardingContainerPoolBalancer(config: WhiskConfig, controllerInstance:
InstanceId)(
+ implicit val actorSystem: ActorSystem,
+ logging: Logging,
+ materializer: ActorMaterializer)
+extends LoadBalancer {
+
+ private implicit val executionContext: ExecutionContext =
actorSystem.dispatcher
+
+ /** Build a cluster of all loadbalancers */
+ val seedNodesProvider = new
StaticSeedNodesProvider(config.controllerSeedNodes, actorSystem.name)
+ val cluster = Cluster(actorSystem)
+ cluster.joinSeedNodes(seedNodesProvider.getSeedNodes())
+
+ /** Used to manage an action for testing invoker health */
+ private val entityStore = WhiskEntityStore.datastore(config)
+
+ /** State related to invocations and throttling */
+ private val activations = TrieMap[ActivationId, ActivationEntry]()
+ private val activationsPerNamespace = TrieMap[UUID, LongAdder]()
+ private val totalActivations = new LongAdder()
+
+ /** State needed for scheduling. */
+ private val schedulingState = ShardingContainerPoolBalancerState()()
+
+ /**
+ * Monitors invoker supervision and the cluster to update the state
sequentially
+ *
+ * All state updates should go through this actor to guarantee, that
`updateState` and `updateCluster` are called
+ * mutually exclusive and not concurrently.
+ */
+ private val monitor = actorSystem.actorOf(Props(new Actor {
+override def preStart(): Unit = {
+ cluster.subscribe(self, classOf[MemberEvent], classOf[ReachabilityEvent])
+}
+
+// all members of the cluster that are available
+var availableMembers = Set.empty[Member]
+
+override def receive: Receive = {
+ case CurrentInvokerPoolState(newState) =>
+schedulingState.updateInvokers(newState)
+
+ // State of the cluster as it is right now
+ case CurrentClusterState(members, _, _, _, _) =>
+availableMembers = members.filter(_.status == MemberStatus.Up)
+schedulingState.updateCluster(availableMembers.size)
+
+ // General lifecycle events and events concerning the reachability of
members. Split-brain is not a huge concern
+ // in this case as only the invoker-threshold is adjusted according to
the perceived cluster-size.
+ // Taking the unreachable member out of the cluster from that
point-of-view results in a better experience
+ // even under split-brain-conditions, as that (in the worst-case)
results in premature overloading of invokers vs.
+ // going into overflow mode prematurely.
+ case event: ClusterDomainEvent =>
+
[GitHub] markusthoemmes commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats
markusthoemmes commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats URL: https://github.com/apache/incubator-openwhisk/pull/3249#issuecomment-363162207 Neaat, thanks for doing this! ? Does this enable any means of integration/automization? I.e.: Flag a PR if it reduces coverage. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh opened a new pull request #3249: #3243 - Collect test code coverage data
chetanmeh opened a new pull request #3249: #3243 - Collect test code coverage data URL: https://github.com/apache/incubator-openwhisk/pull/3249 Adds scoverage plugin to instrument and collect coverage data. Also configures a `aggregateCoverage` task to generate a combined report This PR enables collections of coverage data for tests run in vm and do not hit controller or invoker dockers containers. You can see the coverage stats at [codecov][1] for my fork. With just the lean set (used for travis ci) we get a coverage of ~ 72%. The build time for test phase increased by ~ 2 min (27m 17s to 29m 23s) Codecov integration can be enabled with simple change in `build.sh` for travis. Collecting coverage stats for containers would need [more work](https://github.com/apache/incubator-openwhisk/issues/3243#issuecomment-363160950) ```diff diff --git a/tools/travis/build.sh b/tools/travis/build.sh index 4870acb72a..fdd77b7c67 100755 --- a/tools/travis/build.sh +++ b/tools/travis/build.sh @@ -40,10 +40,12 @@ $ANSIBLE_CMD openwhisk.yml cd $ROOTDIR cat whisk.properties -TERM=dumb ./gradlew :tests:testLean $GRADLE_PROJS_SKIP +TERM=dumb ./gradlew tests:testCoverageLean $GRADLE_PROJS_SKIP :tests:reportCoverage cd $ROOTDIR/ansible $ANSIBLE_CMD logs.yml cd $ROOTDIR tools/build/checkLogs.py logs + +bash <(curl -s https://codecov.io/bash) \ No newline at end of file ``` [1]: https://codecov.io/gh/chetanmeh/incubator-openwhisk/branch/3243-coverage-ci This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on issue #3243: Collect test code coverage data
chetanmeh commented on issue #3243: Collect test code coverage data URL: https://github.com/apache/incubator-openwhisk/issues/3243#issuecomment-363160950 ### Code coverage and Docker execution For capturing coverage stats for tests which hit the controller and invoker docker containers we would need following changes 1. Include the scoverage jars in the docker container and launch script classpath 2. Include the instrumented jars for controller and invoker and ensure that they come before in classpath 3. Provide a writable mount to `/core/controller/build/scoverage` directory (+ for invoker) such that measurement files can be written This can possibly be done by producing 2 extra containers which extend the main containers and have been customized for coverage. Further the ansible scripts would need to pick these containers This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dgrove-oss commented on issue #3219: Kubernetes ContainerFactoryProvider implementation
dgrove-oss commented on issue #3219: Kubernetes ContainerFactoryProvider implementation URL: https://github.com/apache/incubator-openwhisk/pull/3219#issuecomment-363152876 @jcrossley3 - I dug into the PG test failures this morning. The problem is that this PR needs to be rebased to pick up the change from #3063 that moves common/scala/Dockerfile to use alpine linux as a base image. Without that change in base image, it won't go through the PG testing pipeline. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah opened a new pull request #20: Bump npm openwhisk version to pick up latest release.
rabbah opened a new pull request #20: Bump npm openwhisk version to pick up latest release. URL: https://github.com/apache/incubator-openwhisk-runtime-nodejs/pull/20 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dgrove-oss commented on issue #3219: Kubernetes ContainerFactoryProvider implementation
dgrove-oss commented on issue #3219: Kubernetes ContainerFactoryProvider implementation URL: https://github.com/apache/incubator-openwhisk/pull/3219#issuecomment-363152876 @jcrossley3 - I dug into the PG test failures this morning. The problem is that the PR needs to be rebased to pick up the change from #3063 that moves common/scala/Dockerfile to use alpine linux as a base image. Without that change in base image, it won't go through the PG testing pipeline. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jthomas commented on issue #10: Remove redundant file names from exclusion list.
jthomas commented on issue #10: Remove redundant file names from exclusion list. URL: https://github.com/apache/incubator-openwhisk-runtime-swift/pull/10#issuecomment-363152256 Thanks for the comments - I'll add a test case. I found this issue from using Swift with The Serverless Framework. It bundles all local resources into a zip and deploys that, rather the raw source files. Supporting the custom `Package.swift` is definitely a seperate issue. The current Swift runtime does already supporting deploying an archive file with multiple Swift files with the `Package.swift` being ignored. Developers might want to use an archive file to include non-Swift resources (JSON config file, image or other static resources) or split their source code into multiple files. This behaviour is consistent with the other runtimes at the moment. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tymanoj commented on issue #3248: Unable to connect to controller0 while running openwhisk.yml (local deployment)
tymanoj commented on issue #3248: Unable to connect to controller0 while running openwhisk.yml (local deployment) URL: https://github.com/apache/incubator-openwhisk/issues/3248#issuecomment-363140088 Hi, Kindly find required logs in the attached file. [controller0_syslogs.txt](https://github.com/apache/incubator-openwhisk/files/1695926/controller0_syslogs.txt) Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tymanoj opened a new issue #3248: Unable to connect to controller0 while running openwhisk.yml (local deployment)
tymanoj opened a new issue #3248: Unable to connect to controller0 while
running openwhisk.yml (local deployment)
URL: https://github.com/apache/incubator-openwhisk/issues/3248
## Environment details:
* local deployment
* Docker version 17.12.0-ce, build c97c6d6, ubuntu 16.04 LTS, Vagrant
1.8.1, couchdb 2.1.1
## Steps to reproduce the issue:
1. Follow the steps for local development on Ubuntu
2. Use CouchDB as the database
3. The situation will occur while running openwhisk.yml with
ansible-playbook.
4. It waits for the controller for all 12 retries and then gives up.
However, I can see a container running controller on 0.0.0.0:10001->8080/tcp.
## Provide the expected results and outputs:
```
The expected result is to run **sudo ansible-playbook openwhisk.yml**
command without any failures.
```
## Provide the actual results and outputs:
```
TASK [controller : create seed nodes list]
**
Monday 05 February 2018 17:14:58 +0200 (0:00:00.179) 0:00:37.382
***
ok: [controller0] => (item=(0, u'172.17.0.1'))
TASK [controller : (re)start controller]
Monday 05 February 2018 17:14:58 +0200 (0:00:00.210) 0:00:37.592
***
changed: [controller0]
TASK [controller : wait until the Controller in this host is up and running]
Monday 05 February 2018 17:15:02 +0200 (0:00:03.716) 0:00:41.308
***
FAILED - RETRYING: wait until the Controller in this host is up and running
(12 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(11 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(10 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(9 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(8 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(7 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(6 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(5 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(4 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(3 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(2 retries left).
FAILED - RETRYING: wait until the Controller in this host is up and running
(1 retries left).
fatal: [controller0]: FAILED! => {"attempts": 12, "changed": false,
"content": "", "failed": true, "msg": "Status code was not [200]: Request
failed: ", "redirected": false,
"status": -1, "url": "http://172.17.0.1:10001/ping"}
PLAY RECAP
**
controller0: ok=16 changed=3unreachable=0failed=1
kafka0 : ok=8changed=6unreachable=0failed=0
Monday 05 February 2018 17:16:04 +0200 (0:01:02.233) 0:01:43.542
***
===
controller : wait until the Controller in this host is up and running --
62.23s
kafka : (re)start kafka
13.16s
zookeeper : wait until the Zookeeper in this host is up and running -
7.29s
kafka : wait until the kafka server started up --
5.34s
controller : (re)start controller ---
3.72s
zookeeper : (re)start zookeeper -
3.35s
zookeeper : pull the zookeeper:3.4 image
2.24s
kafka : pull the wurstmeister/kafka:0.11.0.1 image --
1.70s
Gathering Facts -
0.61s
controller : copy jmxremote password file ---
0.59s
Gathering Facts -
0.40s
controller : copy jmxremote access file -
0.40s
Gathering Facts -
0.36s
controller : check if whisk_local_whisks with CouchDB exists
0.31s
controller : ensure controller config directory is created with permissions
--- 0.31s
controller : ensure controller log directory is created with permissions ---
0.23s
controller : create seed
[GitHub] rabbah commented on a change in pull request #3202: Support action continuations in the controller
rabbah commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166010563
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] rabbah commented on a change in pull request #3202: Support action continuations in the controller
rabbah commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166010563
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166010182
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] rabbah commented on a change in pull request #3202: Support action continuations in the controller
rabbah commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166009588
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166004339
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166004119
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166003033
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166002577
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166002187
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r166001012
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] ScottChapman commented on issue #1825: Support triggers and rules under package?
ScottChapman commented on issue #1825: Support triggers and rules under package? URL: https://github.com/apache/incubator-openwhisk/issues/1825#issuecomment-363113379 Yea, I think it would be wise to bump the priority of this. First it just feel arbitrary. And second and you start getting more and more integrations it will become more and more problematic for users. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #1825: Support triggers and rules under package?
rabbah commented on issue #1825: Support triggers and rules under package? URL: https://github.com/apache/incubator-openwhisk/issues/1825#issuecomment-363112972 As noted here: https://github.com/apache/incubator-openwhisk/issues/3247#issuecomment-363112438 there are some nuances we need to address, but otherwise allowing a trigger or rule to be nested in a package is plausible. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3247: Packages should also be able to contain Triggers
rabbah commented on issue #3247: Packages should also be able to contain Triggers URL: https://github.com/apache/incubator-openwhisk/issues/3247#issuecomment-363112829 Going to close in favor of #1825. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #1825: Support triggers and rules under package?
rabbah commented on issue #1825: Support triggers and rules under package? URL: https://github.com/apache/incubator-openwhisk/issues/1825#issuecomment-337882339 Yes we should do this. Hasn?t been a priority though with a lot of work on performance and scale out. There are some nuanced though that need to be worked out. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3247: Packages should also be able to contain Triggers
rabbah commented on issue #3247: Packages should also be able to contain Triggers URL: https://github.com/apache/incubator-openwhisk/issues/3247#issuecomment-363112438 Duplicate of https://github.com/apache/incubator-openwhisk/issues/1825. There are some nuances for allowing a trigger and rule in a package. A trigger is usually created with a feed. So would one create a trigger in a package with a feed attached, and then permit bindings of that package? Is a trigger final and hence binding parameters don't extend to an established feed? What if a package with a trigger is shared? These reasons and some historical artifacts of my original implementation is mostly why we haven't added this support yet. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ScottChapman opened a new issue #3247: Packages should also be able to contain Triggers
ScottChapman opened a new issue #3247: Packages should also be able to contain Triggers URL: https://github.com/apache/incubator-openwhisk/issues/3247 I'm sure that this has been discussed at great length, so I apologize for bringing it up as an issue. But it just seems arbitrary to me that Triggers cannot be in packages. Actions and Feeds (which themselves fire Trigger events) can be in Packages as a way to keep related artifacts together. Having Triggers be in the default package means one needs to rely on naming conventions to avoid conflict and to provide clarity. Is there a technical reason why Triggers do not participate in packages? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165991136
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165987836
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165979506
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165979075
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] rabbah commented on a change in pull request #3202: Support action continuations in the controller
rabbah commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165978682
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -65,18 +76,67 @@ protected[actions] trait PrimitiveActions {
/** Database service to get activations. */
protected val activationStore: ActivationStore
+ /** A method that knows how to invoke a sequence of actions. */
+ protected[actions] def invokeSequence(
+user: Identity,
+action: WhiskActionMetaData,
+components: Vector[FullyQualifiedEntityName],
+payload: Option[JsObject],
+waitForOutermostResponse: Option[FiniteDuration],
+cause: Option[ActivationId],
+topmost: Boolean,
+atomicActionsCount: Int)(implicit transid: TransactionId):
Future[(Either[ActivationId, WhiskActivation], Int)]
+
/**
+ * A method that knows how to invoke a single primitive action or a
composition.
+ *
+ * A composition is a kind of sequence of actions that is dynamically
computed.
+ * The execution of a composition is triggered by the invocation of a
conductor action.
+ * A conductor action is an executable action with a defined "conductor"
annotation (the value does not matter).
+ * Sequences cannot be compositions: the "conductor" annotation on a
sequence has no effect.
+ *
+ * A conductor action may either return a final result or a triplet {
action, params, state }.
+ * In the latter case, the specified component action is invoked on the
specified params object.
+ * Upon completion of this action the conductor action is reinvoked with a
payload that combines
+ * the output of the action with the state returned by the previous
conductor invocation.
+ * The composition result is the result of the final conductor invocation in
the chain of invocations.
+ *
+ * The trace of a composition obeys the grammar:
conductorInvocation(componentInvocation conductorInvocation)*
+ *
+ * The activation records for a composition and its components mimic the
activation records of sequences.
+ * They include the same "topmost", "kind", and "causedBy" annotations with
the same semantics.
+ * The activation record for a composition also includes a specific
annotation "conductor" with value true.
+ */
+ protected[actions] def invokeSingleAction(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload: Option[JsObject],
+waitForResponse: Option[FiniteDuration],
+cause: Option[ActivationId])(implicit transid: TransactionId):
Future[Either[ActivationId, WhiskActivation]] = {
+
+if (action.annotations.get(WhiskActivation.conductorAnnotation).isDefined)
{
Review comment:
Truthy ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rabbah commented on a change in pull request #3202: Support action continuations in the controller
rabbah commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165978520
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165978394
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165978286
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165978104
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
+ * @param cause the cause of the composition (activationId of the enclosing
sequence or composition if any)
+ * @param duration the "user" time so far executing the composition (sum of
durations for
+ *all actions invoked so far which is different from the total time
spent executing the composition)
+ * @param maxMemory the maximum memory annotation observed so far for the
conductor action and components
+ * @param state the json state object to inject in the parameter object of
the next conductor invocation
+ * @param accounting the global accounting object used to abort compositions
requiring too many action invocations
+ * @param logs a mutable buffer that is appended with new activation ids as
the composition unfolds
+ * (in contrast with sequences, the logs of a hierarchy of
compositions is not flattened)
+ * @param caller the session object for the parent composition (caller) if
any
+ * @param result a placeholder for returning the result of the composition
invocation (blocking invocation only)
+ */
+ private case class Session(activationId: ActivationId,
+ start: Instant,
+ action: ExecutableWhiskActionMetaData,
+ cause: Option[ActivationId],
+ var duration: Long,
+ var maxMemory: Int,
+ var state: Option[JsObject],
+ accounting: CompositionAccounting,
+ logs: Buffer[ActivationId],
+ caller: Option[Session],
+ result: Option[Promise[Either[ActivationId,
WhiskActivation]]])
+
+ /**
+ * A method that knows how to invoke a composition.
+ *
+ * The method instantiates the session object for the composition, invokes
the conductor action for the composition,
+ * and waits for the composition result (resulting activation) if the
invocation is blocking (up to timeout).
+ *
+ * @param user the identity invoking the action
+ * @param action the conductor action to invoke for the composition
+ * @param payload the dynamic arguments for the activation
+ * @param waitForResponse if not empty, wait upto specified duration for a
response (this is used for blocking activations)
+ * @param cause the activation id that is responsible for this
invoke/activation
+ * @param caller the session object for the caller if any
+ * @param transid a transaction id for logging
+ * @return a promise that completes with one of the following successful
cases:
+ *Right(WhiskActivation) if waiting for a response and response
is ready within allowed duration,
+ *Left(ActivationId) if not waiting for a response, or allowed
duration has elapsed without a result ready
+ * or an error
+ */
+ private def invokeComposition(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload:
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165975020
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -65,18 +76,67 @@ protected[actions] trait PrimitiveActions {
/** Database service to get activations. */
protected val activationStore: ActivationStore
+ /** A method that knows how to invoke a sequence of actions. */
+ protected[actions] def invokeSequence(
+user: Identity,
+action: WhiskActionMetaData,
+components: Vector[FullyQualifiedEntityName],
+payload: Option[JsObject],
+waitForOutermostResponse: Option[FiniteDuration],
+cause: Option[ActivationId],
+topmost: Boolean,
+atomicActionsCount: Int)(implicit transid: TransactionId):
Future[(Either[ActivationId, WhiskActivation], Int)]
+
/**
+ * A method that knows how to invoke a single primitive action or a
composition.
+ *
+ * A composition is a kind of sequence of actions that is dynamically
computed.
+ * The execution of a composition is triggered by the invocation of a
conductor action.
+ * A conductor action is an executable action with a defined "conductor"
annotation (the value does not matter).
+ * Sequences cannot be compositions: the "conductor" annotation on a
sequence has no effect.
+ *
+ * A conductor action may either return a final result or a triplet {
action, params, state }.
+ * In the latter case, the specified component action is invoked on the
specified params object.
+ * Upon completion of this action the conductor action is reinvoked with a
payload that combines
+ * the output of the action with the state returned by the previous
conductor invocation.
+ * The composition result is the result of the final conductor invocation in
the chain of invocations.
+ *
+ * The trace of a composition obeys the grammar:
conductorInvocation(componentInvocation conductorInvocation)*
+ *
+ * The activation records for a composition and its components mimic the
activation records of sequences.
+ * They include the same "topmost", "kind", and "causedBy" annotations with
the same semantics.
+ * The activation record for a composition also includes a specific
annotation "conductor" with value true.
+ */
+ protected[actions] def invokeSingleAction(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload: Option[JsObject],
+waitForResponse: Option[FiniteDuration],
+cause: Option[ActivationId])(implicit transid: TransactionId):
Future[Either[ActivationId, WhiskActivation]] = {
+
+if (action.annotations.get(WhiskActivation.conductorAnnotation).isDefined)
{
Review comment:
In practice, I use the `conductor` annotation to annotate the action with
the json object encoding the composition, so it is not a Boolean value. I could
check that the annotation value is thruthy (not null, zero, '', or undefined).
Thoughts?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165975818
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -139,6 +199,356 @@ protected[actions] trait PrimitiveActions {
}
}
+ /**
+ * Mutable cumulative accounting of what happened during the execution of a
composition.
+ *
+ * Compositions are aborted if the number of action invocations exceeds a
limit.
+ * The permitted max is n component invocations plus 2n+1 conductor
invocations (where n is the actionSequenceLimit).
+ * The max is chosen to permit a sequence with up to n primitive actions.
+ *
+ * NOTE:
+ * A sequence invocation counts as one invocation irrespective of the number
of action invocations in the sequence.
+ * If one component of a composition is also a composition, the caller and
callee share the same accounting object.
+ * The counts are shared between callers and callees so the limit applies
globally.
+ *
+ * @param components the current count of component actions already invoked
+ * @param conductors the current count of conductor actions already invoked
+ */
+ private case class CompositionAccounting(var components: Int = 0, var
conductors: Int = 0)
+
+ /**
+ * A mutable session object to keep track of the execution of one
composition.
+ *
+ * NOTE:
+ * The session object is not shared between callers and callees.
+ * A callee has a reference to the session object for the caller.
+ * This permits the callee to return to the caller when done.
+ *
+ * @param activationId the activationId for the composition (ie the
activation record for the composition)
+ * @param start the start time for the composition
+ * @param action the conductor action responsible for the execution of the
composition
Review comment:
A composition is not permitted to have more than one conductor.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] tardieu commented on a change in pull request #3202: Support action continuations in the controller
tardieu commented on a change in pull request #3202: Support action
continuations in the controller
URL:
https://github.com/apache/incubator-openwhisk/pull/3202#discussion_r165975020
##
File path:
core/controller/src/main/scala/whisk/core/controller/actions/PrimitiveActions.scala
##
@@ -65,18 +76,67 @@ protected[actions] trait PrimitiveActions {
/** Database service to get activations. */
protected val activationStore: ActivationStore
+ /** A method that knows how to invoke a sequence of actions. */
+ protected[actions] def invokeSequence(
+user: Identity,
+action: WhiskActionMetaData,
+components: Vector[FullyQualifiedEntityName],
+payload: Option[JsObject],
+waitForOutermostResponse: Option[FiniteDuration],
+cause: Option[ActivationId],
+topmost: Boolean,
+atomicActionsCount: Int)(implicit transid: TransactionId):
Future[(Either[ActivationId, WhiskActivation], Int)]
+
/**
+ * A method that knows how to invoke a single primitive action or a
composition.
+ *
+ * A composition is a kind of sequence of actions that is dynamically
computed.
+ * The execution of a composition is triggered by the invocation of a
conductor action.
+ * A conductor action is an executable action with a defined "conductor"
annotation (the value does not matter).
+ * Sequences cannot be compositions: the "conductor" annotation on a
sequence has no effect.
+ *
+ * A conductor action may either return a final result or a triplet {
action, params, state }.
+ * In the latter case, the specified component action is invoked on the
specified params object.
+ * Upon completion of this action the conductor action is reinvoked with a
payload that combines
+ * the output of the action with the state returned by the previous
conductor invocation.
+ * The composition result is the result of the final conductor invocation in
the chain of invocations.
+ *
+ * The trace of a composition obeys the grammar:
conductorInvocation(componentInvocation conductorInvocation)*
+ *
+ * The activation records for a composition and its components mimic the
activation records of sequences.
+ * They include the same "topmost", "kind", and "causedBy" annotations with
the same semantics.
+ * The activation record for a composition also includes a specific
annotation "conductor" with value true.
+ */
+ protected[actions] def invokeSingleAction(
+user: Identity,
+action: ExecutableWhiskActionMetaData,
+payload: Option[JsObject],
+waitForResponse: Option[FiniteDuration],
+cause: Option[ActivationId])(implicit transid: TransactionId):
Future[Either[ActivationId, WhiskActivation]] = {
+
+if (action.annotations.get(WhiskActivation.conductorAnnotation).isDefined)
{
Review comment:
In practice, I use the `conductor` annotation to annotate the action with
the json object encoding the composition, so it is not a Boolean value. I could
check that the annotation value is thruthy (not null, zero, '', or undefined).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] csantanapr commented on issue #12: Swift SDK does not support self-signed server certificates.
csantanapr commented on issue #12: Swift SDK does not support self-signed
server certificates.
URL:
https://github.com/apache/incubator-openwhisk-runtime-swift/issues/12#issuecomment-363085463
@jthomas
>In the code above, this won't be the default behaviour. It relies on the
user explictly setting a function parameter, which defaults to false. This
mirrors the behaviour of the JavaScript SDK.
Sorry read your comment again, I think I missed it the first time.
The default behavior for what ever reason today (ie. 3.1.1) _Whisk class API
is `blocking: Bool = true` different from the JavaScript SDK.
https://github.com/apache/incubator-openwhisk-runtime-swift/blob/master/core/swift3.1.1Action/spm-build/_Whisk.swift#L23
```
class func invoke(actionNamed action : String, withParameters params :
[String:Any], blocking: Bool = true) -> [String:Any] {
```
It would be good to have consistency with theJavaScript SDK, but I think is
more important to not break people's code by changing the API when they move
from swift 3 to swift4.
I think it's something to note when we get to building new Library/SDK for
swift4+ both client and server, and maybe a new Class `OpenWhisk`, then I
think will have an opportunity to change the API to invoke
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
