[GitHub] mbehrendt commented on issue #3225: reduce fillInStackTrace expenses due to ArgNormalizer
mbehrendt commented on issue #3225: reduce fillInStackTrace expenses due to ArgNormalizer URL: https://github.com/apache/incubator-openwhisk/pull/3225#issuecomment-360961807 great finding! Thx @starpit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit commented on issue #3225: reduce fillInStackTrace expenses due to ArgNormalizer
starpit commented on issue #3225: reduce fillInStackTrace expenses due to ArgNormalizer URL: https://github.com/apache/incubator-openwhisk/pull/3225#issuecomment-360951987 for future reference, here is an example call stack for one of the two call sites: ``` at java.lang.Throwable.fillInStackTrace(Native Method) at java.lang.Throwable.fillInStackTrace(Throwable.java:783) - locked <0xe2187818> (a spray.json.JsonParser$ParsingException) at java.lang.Throwable.(Throwable.java:265) at java.lang.Exception.(Exception.java:66) at java.lang.RuntimeException.(RuntimeException.java:62) at spray.json.JsonParser$ParsingException.(JsonParser.scala:31) at spray.json.JsonParser.fail(JsonParser.scala:213) at spray.json.JsonParser.value(JsonParser.scala:64) at spray.json.JsonParser.parseJsValue(JsonParser.scala:43) at spray.json.JsonParser$.apply(JsonParser.scala:28) at spray.json.PimpedString.parseJson(package.scala:45) at whisk.core.entity.ArgNormalizer$$anonfun$factory$1.apply(ArgNormalizer.scala:30) at whisk.core.entity.ArgNormalizer$$anonfun$factory$1.apply(ArgNormalizer.scala:30) at scala.util.Try$.apply(Try.scala:192) at whisk.core.entity.ArgNormalizer$class.factory(ArgNormalizer.scala:30) at whisk.core.entity.DocId$.factory(DocInfo.scala:107) at whisk.core.entity.ArgNormalizer$class.apply(ArgNormalizer.scala:45) at whisk.core.entity.DocId$.apply(DocInfo.scala:107) at whisk.core.entity.FullyQualifiedEntityName.toDocId(FullyQualifiedEntityName.scala:51) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit opened a new pull request #3225: reduce fillInStackTrace expenses due to ArgNormalizer
starpit opened a new pull request #3225: reduce fillInStackTrace expenses due to ArgNormalizer URL: https://github.com/apache/incubator-openwhisk/pull/3225 ArgNormalizer uses a try/catch pattern when parsing entity names. in two common cases, the arg to be normalized is known to be a string. we can fast-path this case. i called this `DocId.direct()` the expense of this can be witnessed from call stack samples: you will see `fillInStackTrace` due to exceptions being thrown by spray's json parser. preliminary experiments show that this results in noticeably more stable performance, with p99 latency numbers on echo invocations of: ``` p25 p50 p75 p90 p99 max master 23ms28ms 39ms 58ms 417ms 971ms this PR19ms 22ms 27ms 37ms 99ms586ms ``` test protocol: wrk -t8 -c800 -d60s --latency This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] shazron commented on issue #91: Add documentation for adding the OpenWhisk catalog
shazron commented on issue #91: Add documentation for adding the OpenWhisk catalog URL: https://github.com/apache/incubator-openwhisk-devtools/pull/91#issuecomment-360950979 Thanks @jthomas ! closing this This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] shazron closed pull request #91: Add documentation for adding the OpenWhisk catalog
shazron closed pull request #91: Add documentation for adding the OpenWhisk catalog URL: https://github.com/apache/incubator-openwhisk-devtools/pull/91 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docker-compose/README.md b/docker-compose/README.md index 02e24f9..6c36f3d 100644 --- a/docker-compose/README.md +++ b/docker-compose/README.md @@ -104,6 +104,14 @@ To remove the stopped containers, clean the database files and the temporary fil make destroy ``` +# Add Catalog + +The following command adds the [openwhisk-catalog](https://github.com/apache/incubator-openwhisk-catalog): + +```bash +make add-catalog +``` + # Running a hello-world function Once OpenWhisk is up and running you can execute a `hello-world` function: This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #2833: MesosContainerFactory
tysonnorris commented on a change in pull request #2833: MesosContainerFactory URL: https://github.com/apache/incubator-openwhisk/pull/2833#discussion_r164245116 ## File path: docs/mesos.md ## @@ -0,0 +1,47 @@ +# Mesos Support + +The MesosContainerFactory enables launching action containers within a Mesos cluster. It does not affect the deployment of OpenWhisk components (invoker, controller). + +## Enable + +To enable MesosContainerFactory, use the following TypeSafe Config properties + +| property | required | details | example | +| --- | --- | --- | --- | +| whisk.spi.ContainerFactoryProvider | required | enable the MesosContainerFactory | whisk.core.mesos.MesosContainerFactoryProvider | +| whisk.mesos.master-url | required | mesos master http endpoint to be accessed from the invoker for framework subscription | http://192.168.99.100:5050 | +| whisk.mesos.master-url-public | optional (default to whisk.mesos.master-url) | public facing mesos master http endpoint for exposing logs to cli users | http://192.168.99.100:5050 | +| whisk.mesos.role | optional (default *) | mesos framework role| any string e.g. `openwhisk` | +| whisk.mesos.failover-timeout-seconds | optional (default 0) | how long to wait for the framework to reconnect with the same id before tasks are terminated | see http://mesos.apache.org/documentation/latest/high-availability-framework-guide/ | +| whisk.mesos.mesos-link-log-message | optional (default true) | display a log message with a link to mesos when using the default LogStore (or no log message) | Since logs are not available for invoker to collect from mesos in general, you can either use an alternate LogStore or direct users to the mesos ui | + +To set these properties for your invoker, set the corresponding environment variables e.g. +```properties +CONFIG_whisk_spi_ContainerFactoryProvider=whisk.core.mesos.MesosContainerFactoryProvider +CONFIG_Dwhisk_mesos_masterUrl=http://192.168.99.100:5050 +``` + +## Known Issues + +* logs are not collected from action containers + + For now, the mesos public url will be included in the logs retrieved via the wsk CLI. Once log retrieval from external sources is enabled, logs from mesos containers would have to be routed to the external source, and then retrieved from that source. + +* no HA or failover support (single invoker per cluster) + + Currently the version of mesos-actor in use does not support HA or failover. Failover support is planned to be provided by: + + * multiple invokers running in an Akka cluster + * the mesos framework actor is a singleton within the cluster + * the mesos framework actor is available from the other invoker nodes + * if the node that contains the mesos framework actor fails : + * the actor will be recreated on a separate invoker node + * the actor will resubscribe to mesos scheduler API with the same ID + * the tasks that were previously launched by the actor will be reconciled + * normal operation resumes + Review comment: The invoker relevant pieces will be part of the mesos-actor (client). There is a separate issue with HA for controllers, where seed node discovery needs to be based on mesos for distributed bookeeping data #2992 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #2833: MesosContainerFactory
tysonnorris commented on a change in pull request #2833: MesosContainerFactory
URL:
https://github.com/apache/incubator-openwhisk/pull/2833#discussion_r164244571
##
File path: common/scala/src/main/scala/whisk/core/mesos/MesosTask.scala
##
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.mesos
+
+import akka.actor.ActorRef
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import akka.stream.scaladsl.Source
+import akka.util.ByteString
+import akka.util.Timeout
+import com.adobe.api.platform.runtime.mesos.Bridge
+import com.adobe.api.platform.runtime.mesos.DeleteTask
+import com.adobe.api.platform.runtime.mesos.Running
+import com.adobe.api.platform.runtime.mesos.SubmitTask
+import com.adobe.api.platform.runtime.mesos.TaskDef
+import java.time.LocalDateTime
+import java.time.format.DateTimeFormatter
+import org.apache.mesos.v1.Protos.TaskState
+import org.apache.mesos.v1.Protos.TaskStatus
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.duration._
+import scala.language.postfixOps
+import spray.json._
+import whisk.common.Logging
+import whisk.common.TransactionId
+import whisk.core.containerpool.Container
+import whisk.core.containerpool.ContainerAddress
+import whisk.core.containerpool.ContainerId
+import whisk.core.entity.ByteSize
+import whisk.core.entity.size._
+
+/**
+ * MesosTask implementation of Container.
+ * Differences from DockerContainer include:
+ * - does not launch container using docker cli, but rather a Mesos framework
+ * - does not support pause/resume
+ * - does not support log collection (currently), but does provide a message
indicating logs can be viewed via Mesos UI
+ * (external log collection and retrieval must be enabled via LogStore SPI to
expose logs to wsk cli)
+ */
+case object Environment
+case class CreateContainer(image: String, memory: String, cpuShare: String)
+
+object MesosTask {
+ val taskLaunchTimeout = Timeout(45 seconds)
+ val taskDeleteTimeout = Timeout(30 seconds)
+
+ def create(mesosClientActor: ActorRef,
+ mesosConfig: MesosConfig,
+ taskIdGenerator: () => String,
+ transid: TransactionId,
+ image: String,
+ userProvidedImage: Boolean = false,
+ memory: ByteSize = 256.MB,
+ cpuShares: Int = 0,
+ environment: Map[String, String] = Map(),
+ network: String = "bridge",
+ dnsServers: Seq[String] = Seq(),
+ name: Option[String] = None,
+ parameters: Map[String, Set[String]] = Map())(implicit ec:
ExecutionContext,
+ log: Logging,
+ as: ActorSystem):
Future[Container] = {
+implicit val tid = transid
+
+log.info(this, s"creating task for image ${image}...")
+
+val mesosCpuShares = cpuShares / 1024.0 //convert openwhisk (docker based)
shares to mesos (cpu percentage)
+val mesosRam = memory.toMB.toInt
+
+//TODO: update mesos-actor to support multiple param values for the same
key via Map[String, Set[String]]
+val flatParams = parameters.filter(_._2.nonEmpty).map(e => (e._1 ->
e._2.head))
+val taskId = taskIdGenerator()
+val task = new TaskDef(
+ taskId,
+ name.getOrElse(image), //task name either the indicated name, or else
the image name
+ image,
+ mesosCpuShares,
+ mesosRam,
+ List(8080), //all action containers listen on 8080
+ Some(0), //port at index 0 used for health
+ false,
+ Bridge,
+ flatParams,
+ environment)
+
+val launched: Future[Running] =
+ mesosClientActor.ask(SubmitTask(task))(taskLaunchTimeout).mapTo[Running]
+
+launched.map(taskDetails => {
+ val taskHost = taskDetails.hostname
+ val taskPort = taskDetails.hostports(0)
+ log.info(this, s"launched task with state
${taskDetails.taskStatus.getState} at ${taskHost}:${taskPort}")
+ val containerIp = new ContainerAddress(taskHost, taskPort)
+ val containerId = new ContainerId(taskId);
+ new MesosTask(containerId, containerIp, ec, log,
[GitHub] tysonnorris commented on a change in pull request #2833: MesosContainerFactory
tysonnorris commented on a change in pull request #2833: MesosContainerFactory
URL:
https://github.com/apache/incubator-openwhisk/pull/2833#discussion_r164244356
##
File path:
common/scala/src/main/scala/whisk/core/mesos/MesosContainerFactory.scala
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.mesos
+
+import akka.actor.ActorRef
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import com.adobe.api.platform.runtime.mesos.MesosClient
+import com.adobe.api.platform.runtime.mesos.Subscribe
+import com.adobe.api.platform.runtime.mesos.SubscribeComplete
+import com.adobe.api.platform.runtime.mesos.Teardown
+import java.time.Instant
+import pureconfig.loadConfigOrThrow
+import scala.concurrent.Await
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import whisk.common.Counter
+import whisk.common.Logging
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.WhiskConfig
+import whisk.core.containerpool.Container
+import whisk.core.containerpool.ContainerFactory
+import whisk.core.containerpool.ContainerFactoryProvider
+import whisk.core.entity.ByteSize
+import whisk.core.entity.ExecManifest
+import whisk.core.entity.InstanceId
+import whisk.core.entity.UUID
+
+case class MesosConfig(masterUrl: String,
+ masterPublicUrl: Option[String] = None,
+ role: String = "*",
+ failoverTimeoutSeconds: FiniteDuration = 0.seconds,
+ mesosLinkLogMessage: Boolean = true)
+
+class MesosContainerFactory(config: WhiskConfig,
+actorSystem: ActorSystem,
+logging: Logging,
+parameters: Map[String, Set[String]],
+mesosConfig: MesosConfig =
loadConfigOrThrow[MesosConfig](ConfigKeys.mesos),
+clientFactory: (ActorSystem, MesosConfig) =>
ActorRef = MesosContainerFactory.createClient,
+taskIdGenerator: () => String =
MesosContainerFactory.taskIdGenerator)
+extends ContainerFactory {
+
+ val subscribeTimeout = 30.seconds
+
+ //init mesos framework:
+ implicit val as: ActorSystem = actorSystem
+ implicit val ec: ExecutionContext = actorSystem.dispatcher
+
+ //mesos master url to subscribe the framework to
+ val mesosMaster = mesosConfig.masterUrl
+ //public mesos url where developers can browse logs (till there is way to
delegate log retrieval to an external system)
+ val mesosMasterPublic = mesosConfig.masterPublicUrl
+
+ var isSubscribed = false;
+
+ val mesosClientActor = clientFactory(as, mesosConfig)
+
+ //periodically retry if subscribing did not suceed
+ as.scheduler.schedule(0.seconds, (subscribeTimeout.toSeconds + 10).seconds) {
+if (!isSubscribed) {
+ subscribe()
+}
+ }
+
+ private def subscribe(): Unit = {
+logging.info(this, s"subscribing to mesos master at ${mesosMaster}")
+//subscribe mesos actor to mesos event stream
+//TODO: subscribe failure should make invoker "unhealthy"
+mesosClientActor
+ .ask(Subscribe)(subscribeTimeout)
+ .mapTo[SubscribeComplete]
+ .onComplete({
+case Success(complete) =>
+ isSubscribed = true
+ logging.info(this, s"subscribe completed successfully...${complete}")
+case Failure(e) => logging.error(this, s"subscribe failed...${e}")
+ })
+ }
+
+ override def createContainer(tid: TransactionId,
+ name: String,
+ actionImage: ExecManifest.ImageName,
+ userProvidedImage: Boolean,
+ memory: ByteSize)(implicit config: WhiskConfig,
logging: Logging): Future[Container] = {
+implicit val transid = tid
+val image = if (userProvidedImage) {
+ actionImage.publicImageName
+} else {
+ actionImage.localImageName(config.dockerRegistry,
config.dockerImagePrefix, Some(config.dockerImageTag))
+}
+
+logging.info(this, s"using Mesos to create
[GitHub] tysonnorris commented on a change in pull request #2833: MesosContainerFactory
tysonnorris commented on a change in pull request #2833: MesosContainerFactory
URL:
https://github.com/apache/incubator-openwhisk/pull/2833#discussion_r164244282
##
File path:
common/scala/src/main/scala/whisk/core/mesos/MesosContainerFactory.scala
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.mesos
+
+import akka.actor.ActorRef
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import com.adobe.api.platform.runtime.mesos.MesosClient
+import com.adobe.api.platform.runtime.mesos.Subscribe
+import com.adobe.api.platform.runtime.mesos.SubscribeComplete
+import com.adobe.api.platform.runtime.mesos.Teardown
+import java.time.Instant
+import pureconfig.loadConfigOrThrow
+import scala.concurrent.Await
+import scala.concurrent.ExecutionContext
+import scala.concurrent.Future
+import scala.concurrent.duration._
+import scala.util.Failure
+import scala.util.Success
+import whisk.common.Counter
+import whisk.common.Logging
+import whisk.common.TransactionId
+import whisk.core.ConfigKeys
+import whisk.core.WhiskConfig
+import whisk.core.containerpool.Container
+import whisk.core.containerpool.ContainerFactory
+import whisk.core.containerpool.ContainerFactoryProvider
+import whisk.core.entity.ByteSize
+import whisk.core.entity.ExecManifest
+import whisk.core.entity.InstanceId
+import whisk.core.entity.UUID
+
+case class MesosConfig(masterUrl: String,
+ masterPublicUrl: Option[String] = None,
+ role: String = "*",
+ failoverTimeoutSeconds: FiniteDuration = 0.seconds,
+ mesosLinkLogMessage: Boolean = true)
+
+class MesosContainerFactory(config: WhiskConfig,
+actorSystem: ActorSystem,
+logging: Logging,
+parameters: Map[String, Set[String]],
+mesosConfig: MesosConfig =
loadConfigOrThrow[MesosConfig](ConfigKeys.mesos),
+clientFactory: (ActorSystem, MesosConfig) =>
ActorRef = MesosContainerFactory.createClient,
+taskIdGenerator: () => String =
MesosContainerFactory.taskIdGenerator)
+extends ContainerFactory {
+
+ val subscribeTimeout = 30.seconds
+
+ //init mesos framework:
+ implicit val as: ActorSystem = actorSystem
+ implicit val ec: ExecutionContext = actorSystem.dispatcher
+
+ //mesos master url to subscribe the framework to
+ val mesosMaster = mesosConfig.masterUrl
+ //public mesos url where developers can browse logs (till there is way to
delegate log retrieval to an external system)
+ val mesosMasterPublic = mesosConfig.masterPublicUrl
+
+ var isSubscribed = false;
+
+ val mesosClientActor = clientFactory(as, mesosConfig)
+
+ //periodically retry if subscribing did not suceed
+ as.scheduler.schedule(0.seconds, (subscribeTimeout.toSeconds + 10).seconds) {
+if (!isSubscribed) {
+ subscribe()
+}
+ }
+
+ private def subscribe(): Unit = {
+logging.info(this, s"subscribing to mesos master at ${mesosMaster}")
+//subscribe mesos actor to mesos event stream
+//TODO: subscribe failure should make invoker "unhealthy"
+mesosClientActor
+ .ask(Subscribe)(subscribeTimeout)
+ .mapTo[SubscribeComplete]
+ .onComplete({
+case Success(complete) =>
+ isSubscribed = true
+ logging.info(this, s"subscribe completed successfully...${complete}")
+case Failure(e) => logging.error(this, s"subscribe failed...${e}")
+ })
+ }
+
+ override def createContainer(tid: TransactionId,
+ name: String,
+ actionImage: ExecManifest.ImageName,
+ userProvidedImage: Boolean,
+ memory: ByteSize)(implicit config: WhiskConfig,
logging: Logging): Future[Container] = {
+implicit val transid = tid
+val image = if (userProvidedImage) {
+ actionImage.publicImageName
+} else {
+ actionImage.localImageName(config.dockerRegistry,
config.dockerImagePrefix, Some(config.dockerImageTag))
+}
+
+logging.info(this, s"using Mesos to create
[GitHub] pritidesai opened a new pull request #699: WIP: adding API Gateway functionality
pritidesai opened a new pull request #699: WIP: adding API Gateway functionality URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/699 This is still in progress and way far from completion. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr closed pull request #3222: Correct the names of two annotations.
csantanapr closed pull request #3222: Correct the names of two annotations.
URL: https://github.com/apache/incubator-openwhisk/pull/3222
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/annotations.md b/docs/annotations.md
index 6b0ab8d9c4..d92bf3346c 100644
--- a/docs/annotations.md
+++ b/docs/annotations.md
@@ -65,8 +65,8 @@ Additionally for sequence related activations, the system
will generate the foll
Lastly, and in order to provide you with some performance transparency,
activations also record:
-* `wait`: the time spent waiting in the internal OpenWhisk system. This is
roughly the time spent between the controller receiving the activation request
and when the invoker provisioned a container for the action. This value is
currently only present for non-sequence related activations. For sequences,
this information can be derived from the `topmost` sequence activation record.
-* `init`: the time spent initializing the function. If this value is present,
the action required initialization and represents a cold start. A warm
activation will skip initialization, and in this case, the annotation is not
generated.
+* `waitTime`: the time spent waiting in the internal OpenWhisk system. This is
roughly the time spent between the controller receiving the activation request
and when the invoker provisioned a container for the action. This value is
currently only present for non-sequence related activations. For sequences,
this information can be derived from the `topmost` sequence activation record.
+* `initTime`: the time spent initializing the function. If this value is
present, the action required initialization and represents a cold start. A warm
activation will skip initialization, and in this case, the annotation is not
generated.
An example of these annotations as they would appear in an activation record
is shown below.
@@ -77,7 +77,7 @@ An example of these annotations as they would appear in an
activation record is
"value": "guest/echo"
},
{
-"key": "wait",
+"key": "waitTime",
"value": 66
},
{
@@ -85,7 +85,7 @@ An example of these annotations as they would appear in an
activation record is
"value": "nodejs:6"
},
{
-"key": "init",
+"key": "initTime",
"value": 50
},
{
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on issue #170: WIP: Initial code checkin for path parameter support
dubeejw commented on issue #170: WIP: Initial code checkin for path parameter support URL: https://github.com/apache/incubator-openwhisk-cli/pull/170#issuecomment-360885648 @jessealva, ready for review? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mdeuser commented on a change in pull request #3187: reduce rule activation records
mdeuser commented on a change in pull request #3187: reduce rule activation
records
URL:
https://github.com/apache/incubator-openwhisk/pull/3187#discussion_r164198480
##
File path: core/controller/src/main/scala/whisk/core/controller/Triggers.scala
##
@@ -122,72 +126,135 @@ trait WhiskTriggersApi extends WhiskCollectionAPI {
version = trigger.version,
duration = None)
- logging.info(this, s"[POST] trigger activated, writing activation
record to datastore: $triggerActivationId")
- WhiskActivation.put(activationStore, triggerActivation) recover {
-case t =>
- logging.error(this, s"[POST] storing trigger activation
$triggerActivationId failed: ${t.getMessage}")
- }
-
- val url = Uri(s"http://localhost:${whiskConfig.servicePort};)
-
- trigger.rules.map {
-_.filter {
- case (ruleName, rule) => rule.status == Status.ACTIVE
-} foreach {
- case (ruleName, rule) =>
-val ruleActivationId = activationIdFactory.make()
-val ruleActivation = WhiskActivation(
- namespace = user.namespace.toPath, // all activations should
end up in the one space regardless trigger.namespace,
- ruleName.name,
- user.subject,
- ruleActivationId,
- Instant.now(Clock.systemUTC()),
- Instant.EPOCH,
- cause = Some(triggerActivationId),
- response = ActivationResponse.success(),
- version = trigger.version,
- duration = None)
-WhiskActivation.put(activationStore, ruleActivation) recover {
- case t =>
-logging.error(this, s"[POST] storing rule activation
$ruleActivationId failed: ${t.getMessage}")
-}
-
-val actionNamespace = rule.action.path.root.asString
-val actionPath = {
- rule.action.path.relativePath.map { pkg =>
-(Path.SingleSlash + pkg.namespace) /
rule.action.name.asString
- } getOrElse {
-Path.SingleSlash + rule.action.name.asString
- }
-}.toString
-
-val actionUrl = Path("/api/v1") / "namespaces" /
actionNamespace / "actions"
-val request = HttpRequest(
- method = POST,
- uri = url.withPath(actionUrl + actionPath),
- headers =
-
List(Authorization(BasicHttpCredentials(user.authkey.uuid.asString,
user.authkey.key.asString))),
- entity = HttpEntity(MediaTypes.`application/json`,
args.getOrElse(JsObject()).compactPrint))
-
-Http().singleRequest(request).map {
- response =>
-response.status match {
- case OK | Accepted =>
-Unmarshal(response.entity).to[JsObject].map { a =>
- logging.info(this, s"${rule.action} activated
${a.fields("activationId")}")
-}
- case NotFound =>
-response.discardEntityBytes()
-logging.info(this, s"${rule.action} failed, action not
found")
- case _ =>
-Unmarshal(response.entity).to[String].map { error =>
- logging.warn(this, s"${rule.action} failed due to
$error")
+ // List of active rules associated with the trigger
+ val activeRules = trigger.rules map { _.filter((r) => r._2.status ==
Status.ACTIVE) } getOrElse Map.empty
+
+ if (activeRules.nonEmpty) {
+val args = trigger.parameters.merge(payload)
+
+// Iterate through each active rule; invoking each mapped action
+val actionLogList = activeRules
+ .map {
+case (ruleName, rule) =>
+ // Build the url to invoke an action mapped to the rule
+ val actionNamespace = rule.action.path.root.asString
+ val actionPath = {
+rule.action.path.relativePath.map { pkg =>
+ (Path.SingleSlash + pkg.namespace) /
rule.action.name.asString
+} getOrElse {
+ Path.SingleSlash + rule.action.name.asString
+}
+ }.toString
+
+ val actionUrl = Path("/api/v1") / "namespaces" /
actionNamespace / "actions"
+ val request = HttpRequest(
+method = POST,
+uri = url.withPath(actionUrl + actionPath),
+headers =
+
List(Authorization(BasicHttpCredentials(user.authkey.uuid.asString,
user.authkey.key.asString))),
+entity =
[GitHub] dubeejw closed pull request #3223: Remove kafka related debug message
dubeejw closed pull request #3223: Remove kafka related debug message
URL: https://github.com/apache/incubator-openwhisk/pull/3223
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/common/scala/src/main/scala/whisk/connector/kafka/KafkaProducerConnector.scala
b/common/scala/src/main/scala/whisk/connector/kafka/KafkaProducerConnector.scala
index 47dce3a2bb..7b0977b825 100644
---
a/common/scala/src/main/scala/whisk/connector/kafka/KafkaProducerConnector.scala
+++
b/common/scala/src/main/scala/whisk/connector/kafka/KafkaProducerConnector.scala
@@ -50,8 +50,6 @@ class KafkaProducerConnector(kafkahosts: String,
override def send(topic: String, msg: Message, retry: Int = 2):
Future[RecordMetadata] = {
implicit val transid = msg.transid
val record = new ProducerRecord[String, String](topic, "messages",
msg.serialize)
-
-logging.debug(this, s"sending to topic '$topic' msg '$msg'")
val produced = Promise[RecordMetadata]()
producer.send(record, new Callback {
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on issue #3223: Remove kafka related debug message
dubeejw commented on issue #3223: Remove kafka related debug message URL: https://github.com/apache/incubator-openwhisk/pull/3223#issuecomment-360818433 PG3 1744 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr closed pull request #236: disable consumer after 5 failed attempts to connect to kafka brokers
csantanapr closed pull request #236: disable consumer after 5 failed attempts
to connect to kafka brokers
URL: https://github.com/apache/incubator-openwhisk-package-kafka/pull/236
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/provider/consumer.py b/provider/consumer.py
index f28c524..d80ad6b 100644
--- a/provider/consumer.py
+++ b/provider/consumer.py
@@ -131,6 +131,8 @@ def __init__(self, trigger, params, sharedDictionary):
self.brokers = params["brokers"]
self.topic = params["topic"]
+self.authErrors = 0
+
self.sharedDictionary = sharedDictionary
if 'status' in params and params['status']['active'] == False:
@@ -253,7 +255,8 @@ def __createConsumer(self):
'group.id': self.trigger,
'default.topic.config': {'auto.offset.reset':
'latest'},
'enable.auto.commit': False,
-'api.version.request': True
+'api.version.request': True,
+'error_cb': self.__error_callback
}
if self.isMessageHub:
@@ -445,3 +448,12 @@ def __encodeKeyIfNeeded(self, key):
logging.debug('[{}] Returning un-encoded message'.format(self.trigger))
return key
+
+def __error_callback(self, error):
+logging.warning(error)
+if error.code() == KafkaError._AUTHENTICATION:
+self.authErrors = self.authErrors + 1
+if self.authErrors > 5:
+self.setDesiredState(Consumer.State.Disabled)
+message = 'Automatically disabled trigger. Consumer failed to
authentication with broker(s) more than 5 times with apikey
{}:{}'.format(self.username, self.password)
+self.database.disableTrigger(self.trigger, 403, message)
diff --git a/provider/database.py b/provider/database.py
index 77dd4c7..68ee8a3 100644
--- a/provider/database.py
+++ b/provider/database.py
@@ -58,7 +58,7 @@ def destroy(self):
self.client.disconnect()
self.client = None
-def disableTrigger(self, triggerFQN, status_code):
+def disableTrigger(self, triggerFQN, status_code, message='Automatically
disabled after receiving a {} status code when firing the trigger.'):
try:
document = self.database[triggerFQN]
@@ -71,7 +71,7 @@ def disableTrigger(self, triggerFQN, status_code):
'reason': {
'kind': 'AUTO',
'statusCode': status_code,
-'message': 'Automatically disabled after receiving a
{} status code when firing the trigger.'.format(status_code)
+'message': message.format(status_code)
}
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] abaruni commented on issue #237: Create test for disabling triggers when consumers cannot connect to brokers
abaruni commented on issue #237: Create test for disabling triggers when consumers cannot connect to brokers URL: https://github.com/apache/incubator-openwhisk-package-kafka/issues/237#issuecomment-360860272 fix was added in https://github.com/apache/incubator-openwhisk-package-kafka/pull/236 to shutdown consumers and disable triggers after 5 failed attempts to connect to brokers. need a test for this This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] abaruni opened a new issue #237: Create test for disabling triggers when consumers cannot connect to brokers
abaruni opened a new issue #237: Create test for disabling triggers when consumers cannot connect to brokers URL: https://github.com/apache/incubator-openwhisk-package-kafka/issues/237 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #236: disable consumer after 5 failed attempts to connect to kafka brokers
csantanapr commented on issue #236: disable consumer after 5 failed attempts to connect to kafka brokers URL: https://github.com/apache/incubator-openwhisk-package-kafka/pull/236#issuecomment-360857349 @dubeejw can you review ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] abaruni opened a new pull request #236: disable consumer after 5 failed attempts to connect to kafka brokers
abaruni opened a new pull request #236: disable consumer after 5 failed attempts to connect to kafka brokers URL: https://github.com/apache/incubator-openwhisk-package-kafka/pull/236 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #5: Add the support to verify the apache licensing header for all the source code
houshengbo commented on issue #5: Add the support to verify the apache licensing header for all the source code URL: https://github.com/apache/incubator-openwhisk-release/issues/5#issuecomment-360845820 @daisy-ycguo I think this tool works functionally similar to what is in our utilities. @mrutkows It looks like we can use this tool in our release process. We have two options to verify the headers. BTW, if we are in favor of apache tools, do we like to add RAT into openwhisk utilities? (low priority) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #3: Use Apache Tentacles to create a LICENSE and NOTICE report
houshengbo commented on issue #3: Use Apache Tentacles to create a LICENSE and NOTICE report URL: https://github.com/apache/incubator-openwhisk-release/issues/3#issuecomment-360844780 LoL. This is what we can do: we add the NOTICE and LICENSE, package them into jars, or whatever zips, launch them into staging. Use this tool to verify the NOTICE and LICENSE later. What if we run the staging repo service ocally, so that we can access via localhost. Still unable to use this tool? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #3: Use Apache Tentacles to create a LICENSE and NOTICE report
houshengbo commented on issue #3: Use Apache Tentacles to create a LICENSE and NOTICE report URL: https://github.com/apache/incubator-openwhisk-release/issues/3#issuecomment-360844780 @daisy-ycguo LoL. This is what we can do: we add the NOTICE and LICENSE, package them into jars, or whatever zips, launch them into staging. Use this tool to verify the NOTICE and LICENSE later. What if we run the staging repo service ocally, so that we can access via localhost. Still unable to use this tool? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows opened a new pull request #698: WIP: Test: remove functions from verifier.go to confirm no test impact.
mrutkows opened a new pull request #698: WIP: Test: remove functions from verifier.go to confirm no test impact. URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/698 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo commented on issue #12: Add the license and NOTICE file
houshengbo commented on issue #12: Add the license and NOTICE file URL: https://github.com/apache/incubator-openwhisk-release/issues/12#issuecomment-360843823 @daisy-ycguo Interesting. Will these two files be static or dynamic from release to release? If it is static, we put them in each repo. If it is dynamic, we need to find out how to when and how to revise them. IMO, they seem to be static. Thoughts? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dgrove-oss commented on issue #3219: Kubernetes ContainerFactoryProvider implementation
dgrove-oss commented on issue #3219: Kubernetes ContainerFactoryProvider implementation URL: https://github.com/apache/incubator-openwhisk/pull/3219#issuecomment-360840373 Happy to report this worked for me (once I figured out the one magic config variable to pass on the command line for my kube environment...). I have a WIP PR in the deploy-kube that adds the kubernetes yml files to deploy an invoker that uses this new pool. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #2968: add additional metrics and logs
rabbah commented on a change in pull request #2968: add additional metrics and
logs
URL:
https://github.com/apache/incubator-openwhisk/pull/2968#discussion_r164162134
##
File path:
core/controller/src/main/scala/whisk/core/loadBalancer/ContainerPoolBalancer.scala
##
@@ -158,6 +160,11 @@ class ContainerPoolBalancer(config: WhiskConfig,
instance: InstanceId)(implicit
processCompletion(Left(activationId), transid, forced = true,
invoker = invokerName)
}
+transid.mark(
+ this,
+ LoggingMarkers.LOADBALANCER_ACTIVATION_START(namespaceId.asString),
+ s"loadbalancer: activation started for namespace $namespaceId")
Review comment:
do we need this vs
```
val start = transid.started(
this,
LoggingMarkers.CONTROLLER_KAFKA,
s"posting topic '$topic' with activation id '${msg.activationId}'",
logLevel = InfoLevel)
```
does it make sense to consolidate the two?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dgrove-oss opened a new pull request #155: initial config files to use kubernetes container pool
dgrove-oss opened a new pull request #155: initial config files to use kubernetes container pool URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/155 config files for deploying an invoker that uses the k8s container pool of https://github.com/apache/incubator-openwhisk/pull/3219 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #2968: add additional metrics and logs
rabbah commented on a change in pull request #2968: add additional metrics and
logs
URL:
https://github.com/apache/incubator-openwhisk/pull/2968#discussion_r164162134
##
File path:
core/controller/src/main/scala/whisk/core/loadBalancer/ContainerPoolBalancer.scala
##
@@ -158,6 +160,11 @@ class ContainerPoolBalancer(config: WhiskConfig,
instance: InstanceId)(implicit
processCompletion(Left(activationId), transid, forced = true,
invoker = invokerName)
}
+transid.mark(
+ this,
+ LoggingMarkers.LOADBALANCER_ACTIVATION_START(namespaceId.asString),
+ s"loadbalancer: activation started for namespace $namespaceId")
Review comment:
do we need this vs
```
val start = transid.started(
this,
LoggingMarkers.CONTROLLER_KAFKA,
s"posting topic '$topic' with activation id '${msg.activationId}'",
logLevel = InfoLevel)
```
?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rabbah commented on a change in pull request #2968: add additional metrics and logs
rabbah commented on a change in pull request #2968: add additional metrics and
logs
URL:
https://github.com/apache/incubator-openwhisk/pull/2968#discussion_r164161468
##
File path: common/scala/src/main/scala/whisk/common/Logging.scala
##
@@ -248,6 +251,8 @@ object LoggingMarkers {
val INVOKER_ACTIVATION = LogMarkerToken(invoker, activation, start)
def INVOKER_DOCKER_CMD(cmd: String) = LogMarkerToken(invoker,
s"docker.$cmd", start)
def INVOKER_RUNC_CMD(cmd: String) = LogMarkerToken(invoker, s"runc.$cmd",
start)
+ def INVOKER_CONTAINER_START(actionName: String, namespaceName: String,
containerState: String) =
+LogMarkerToken(invoker,
s"container_start_${containerState}_${namespaceName}_$actionName", count)
Review comment:
same comment re param typing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rabbah commented on a change in pull request #2968: add additional metrics and logs
rabbah commented on a change in pull request #2968: add additional metrics and
logs
URL:
https://github.com/apache/incubator-openwhisk/pull/2968#discussion_r164161348
##
File path: common/scala/src/main/scala/whisk/common/Logging.scala
##
@@ -238,6 +239,8 @@ object LoggingMarkers {
// Check invoker healthy state from loadbalancer
val LOADBALANCER_INVOKER_OFFLINE = LogMarkerToken(loadbalancer,
"invokerOffline", count)
val LOADBALANCER_INVOKER_UNHEALTHY = LogMarkerToken(loadbalancer,
"invokerUnhealthy", count)
+ def LOADBALANCER_ACTIVATION_START(namespaceId: String) =
Review comment:
should this take a string type than string? entity name or identity perhaps.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] mhenke1 commented on a change in pull request #2968: add additional metrics and logs
mhenke1 commented on a change in pull request #2968: add additional metrics and
logs
URL:
https://github.com/apache/incubator-openwhisk/pull/2968#discussion_r164143665
##
File path:
core/invoker/src/main/scala/whisk/core/containerpool/ContainerPool.scala
##
@@ -78,27 +73,56 @@ class ContainerPool(childFactory: ActorRefFactory =>
ActorRef,
}
}
+ def logContainerStart(r: Run, containerState: String): Unit = {
+val namespaceName = r.msg.user.namespace.name
+val actionName = r.action.name.name
+val activationId = r.msg.activationId.toString
+
+r.msg.transid.mark(
+ this,
+ LoggingMarkers.INVOKER_CONTAINER_START(actionName, namespaceName,
containerState),
+ s"containerStart containerState: $containerState action: $actionName
namespace: $namespaceName activationId: $activationId",
+ akka.event.Logging.InfoLevel)
+ }
+
def receive: Receive = {
// A job to run on a container
case r: Run =>
val container = if (busyPool.size < maxActiveContainers) {
+
// Schedule a job to a warm container
ContainerPool
.schedule(r.action, r.msg.user.namespace, freePool)
+ .map(container => {
Review comment:
@markusthoemmes @rabbah I have moved the logging to the end of the method. I
have used a simple tuple to transport the container state.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] starpit commented on issue #3220: Move log message into a closure to defer string interpolation.
starpit commented on issue #3220: Move log message into a closure to defer string interpolation. URL: https://github.com/apache/incubator-openwhisk/pull/3220#issuecomment-360816253 i discovered this by looking at stack traces, after a bunch of other tuning opportunities. i suspect that the benefit of this will grow, as we start shaving off other, larger, inefficiencies. right now, on my 44-way system, for echo invokes, i'm seeing no major difference, under high load, in either throughput, p25, p50, p90, p99, or max. there is a small benefit in p25, under light load. master, for reference, gives me 2500 echo invokes per second, which is pretty far short of the 7000 or so i was getting after a bunch of other changes. this is using the defaults for all settings, except specifying the use of `runc`, and upping the limits. i'll start portioning in some other changes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on issue #3223: Remove kafka related debug message
dubeejw commented on issue #3223: Remove kafka related debug message URL: https://github.com/apache/incubator-openwhisk/pull/3223#issuecomment-360818433 PG3 1744 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on issue #3223: Remove kafka related debug message
dubeejw commented on issue #3223: Remove kafka related debug message URL: https://github.com/apache/incubator-openwhisk/pull/3223#issuecomment-360818433 PG3 1743 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mhenke1 opened a new pull request #3223: Remove log message
mhenke1 opened a new pull request #3223: Remove log message URL: https://github.com/apache/incubator-openwhisk/pull/3223 With enhanced kafka message sizes this debug message writes an enormous amount of data. Since enough information is printed out in error cases, we can simply remove it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit commented on issue #3220: Move log message into a closure to defer string interpolation.
starpit commented on issue #3220: Move log message into a closure to defer string interpolation. URL: https://github.com/apache/incubator-openwhisk/pull/3220#issuecomment-360816253 i discovered this by looking at stack traces, after a bunch of other tuning opportunities. i suspect that the benefit of this will grow, as we start shaving off other, larger, inefficiencies. right now, on my 44-way system, for echo invokes, i'm seeing no major difference, under high load, in either throughput, p25, p50, p90, p99, or max. there is a small benefit in p25, under light load. master, for reference, gives me 2500 echo invokes per second, which is pretty far short of the 7000 or so i was getting after a bunch of other changes. this is using the defaults for all settings, except specifying the use of `runc`, and upping the limits. i'll start portioning in some other changes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit commented on issue #3220: Move log message into a closure to defer string interpolation.
starpit commented on issue #3220: Move log message into a closure to defer string interpolation. URL: https://github.com/apache/incubator-openwhisk/pull/3220#issuecomment-360816253 i discovered this by looking at stack traces, after a bunch of other tuning opportunities. i suspect that the benefit of this will grow, as we start shaving off other, larger, inefficiencies. right now, on my 44-way system, for echo invokes, i'm seeing no major difference in either throughput, p25, p50, p90, p99, or max. there is a small benefit in p25, under light load. master, for reference, gives me 2500 echo invokes per second, which is pretty far short of the 7000 or so i was getting after a bunch of other changes. this is using the defaults for all settings, except specifying the use of `runc`, and upping the limits. i'll start portioning in some other changes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eweiter opened a new issue #255: Feature Request: Consistency across sample parameter names for actions
eweiter opened a new issue #255: Feature Request: Consistency across sample parameter names for actions URL: https://github.com/apache/incubator-openwhisk-catalog/issues/255 A while back Ioana had created this issue in the IBM Cloud Functions UI repo. I'm moving it here because I feel it was invalid for it to be in that previous repo. " I played with the UI to create a sequence and also run actions (sequences or individual/atomic actions). Depending on the context, there are several names that are used for the sample parameters (e.g., params, message, arg). These are usually different than the parameters that the actions in the sample catalog expects. This inconsistency makes it hard to run some examples without touching code or modifying the params that are passed in. It would be nice if there was consistency. PS: This issue also involves the catalog sample actions. For example, I can't chain echo with count just out of the box because of discrepancies in the params/payload naming/schema." This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo opened a new issue #212: Is there is reverse populate command opposite to ./gradlew goVendor
houshengbo opened a new issue #212: Is there is reverse populate command opposite to ./gradlew goVendor URL: https://github.com/apache/incubator-openwhisk-cli/issues/212 After running ./gradlew goVendor, we able to build binaries with native go command. However, we tried to run ./gradlew goTest, it did not run any tests. Do we need to un-populate vendors? Do we have a command to reverse ./gradlew goVendor? @jonpspri ``` ./gradlew goTest > Configure project : Found go 1.8 in /usr/local/go/bin/go, use it. > Task :goPrepare Found global GOPATH: /Users/vincent/go. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo opened a new issue #211: How shall we deal with the traditional commands: ./gradlew test and ./gradlew build
houshengbo opened a new issue #211: How shall we deal with the traditional commands: ./gradlew test and ./gradlew build URL: https://github.com/apache/incubator-openwhisk-cli/issues/211 The question is what is our suggestions to deal with these commands, ./gradlew test and ./gradlew build,? If we are still in favor of them, can we overwrite them by calling the current ./gradlew goTest and ./gradlew compile. If we are not in favor of them, is there a way to remove or inhibit them? @jonpspri Could you look at this question, as the expert of gogradle? Thx. When we run the commands, ./gradlew test and ./gradlew build, brought in by gogradle plugin, the test cases under openwhisk/test will be running as below: ``` Step 1/3 : FROM dockerskeleton ---> b87a7e46c988 Step 2/3 : ENV FLASK_PROXY_PORT 8080 ---> Using cache ---> f6a1758b06ff Step 3/3 : CMD ["/bin/bash", "-c", "tail -f /dev/null"] ---> Using cache ---> c7a6360cc3aa Successfully built c7a6360cc3aa Successfully tagged badproxy:latest > Task :tests:dat:blackbox:badproxy:distDocker Fri Jan 26 09:59:23 EST 2018: Executing 'docker build -t badproxy /Users/vincent/workspace/openwhisk/tests/dat/blackbox/badproxy' Building 'badproxy' took 0.168 seconds > Task :tests:dat:blackbox:badproxy:tagImage Fri Jan 26 09:59:23 EST 2018: Executing 'docker tag badproxy whisk/badproxy:latest' <-> 66% EXECUTING [9s] > :clitests:test > 0 tests completed > :clitests:test > Executing test actionContainers.ActionProxyContainerTests ``` Test cases of scala in openwhisk-cli still depends on the scala code under openwhisk/test. I think that is the reason for this behavior. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah opened a new pull request #3222: Correct the names of two annotations.
rabbah opened a new pull request #3222: Correct the names of two annotations. URL: https://github.com/apache/incubator-openwhisk/pull/3222 the annotations were renamed to add a `Time` suffix. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes closed pull request #3221: Fix overload of invoker.
markusthoemmes closed pull request #3221: Fix overload of invoker.
URL: https://github.com/apache/incubator-openwhisk/pull/3221
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core/controller/src/main/scala/whisk/core/loadBalancer/ContainerPoolBalancer.scala
b/core/controller/src/main/scala/whisk/core/loadBalancer/ContainerPoolBalancer.scala
index f2614e0ca2..aed332e95b 100644
---
a/core/controller/src/main/scala/whisk/core/loadBalancer/ContainerPoolBalancer.scala
+++
b/core/controller/src/main/scala/whisk/core/loadBalancer/ContainerPoolBalancer.scala
@@ -43,9 +43,9 @@ import scala.util.{Failure, Success}
case class LoadbalancerConfig(blackboxFraction: Double, invokerBusyThreshold:
Int)
-class ContainerPoolBalancer(config: WhiskConfig, instance:
InstanceId)(implicit val actorSystem: ActorSystem,
-
logging: Logging,
-
materializer: ActorMaterializer)
+class ContainerPoolBalancer(config: WhiskConfig, controllerInstance:
InstanceId)(implicit val actorSystem: ActorSystem,
+
logging: Logging,
+
materializer: ActorMaterializer)
extends LoadBalancer {
private val lbConfig =
loadConfigOrThrow[LoadbalancerConfig](ConfigKeys.loadbalancer)
@@ -219,7 +219,7 @@ class ContainerPoolBalancer(config: WhiskConfig, instance:
InstanceId)(implicit
private val invokerPool = {
// Do not create the invokerPool if it is not possible to create the
health test action to recover the invokers.
InvokerPool
- .healthAction(instance)
+ .healthAction(controllerInstance)
.map {
// Await the creation of the test action; on failure, this will abort
the constructor which should
// in turn abort the startup of the controller.
@@ -233,9 +233,9 @@ class ContainerPoolBalancer(config: WhiskConfig, instance:
InstanceId)(implicit
val maxPingsPerPoll = 128
val pingConsumer =
- messagingProvider.getConsumer(config, s"health${instance.toInt}",
"health", maxPeek = maxPingsPerPoll)
+ messagingProvider.getConsumer(config,
s"health${controllerInstance.toInt}", "health", maxPeek = maxPingsPerPoll)
val invokerFactory = (f: ActorRefFactory, invokerInstance: InstanceId) =>
- f.actorOf(InvokerActor.props(invokerInstance, instance))
+ f.actorOf(InvokerActor.props(invokerInstance, controllerInstance))
actorSystem.actorOf(
InvokerPool.props(invokerFactory, (m, i) =>
sendActivationToInvoker(messageProducer, m, i), pingConsumer))
@@ -248,7 +248,11 @@ class ContainerPoolBalancer(config: WhiskConfig, instance:
InstanceId)(implicit
val maxActiveAcksPerPoll = 128
val activeAckPollDuration = 1.second
private val activeAckConsumer =
-messagingProvider.getConsumer(config, "completions",
s"completed${instance.toInt}", maxPeek = maxActiveAcksPerPoll)
+messagingProvider.getConsumer(
+ config,
+ "completions",
+ s"completed${controllerInstance.toInt}",
+ maxPeek = maxActiveAcksPerPoll)
val activationFeed = actorSystem.actorOf(Props {
new MessageFeed(
"activeack",
@@ -299,7 +303,7 @@ class ContainerPoolBalancer(config: WhiskConfig, instance:
InstanceId)(implicit
val invokersToUse = if (action.exec.pull) blackboxInvokers(invokers)
else managedInvokers(invokers)
val invokersWithUsage = invokersToUse.view.map {
// Using a view defers the comparably expensive lookup to actual
access of the element
- case invoker => (invoker.id, invoker.status,
currentActivations.getOrElse(instance.toString, 0))
+ case invoker => (invoker.id, invoker.status,
currentActivations.getOrElse(invoker.id.toString, 0))
}
ContainerPoolBalancer.schedule(invokersWithUsage,
lbConfig.invokerBusyThreshold, hash) match {
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cbickel commented on a change in pull request #3221: Fix overload of invoker.
cbickel commented on a change in pull request #3221: Fix overload of invoker.
URL:
https://github.com/apache/incubator-openwhisk/pull/3221#discussion_r164108239
##
File path:
core/controller/src/main/scala/whisk/core/loadBalancer/ContainerPoolBalancer.scala
##
@@ -299,7 +303,7 @@ class ContainerPoolBalancer(config: WhiskConfig, instance:
InstanceId)(implicit
val invokersToUse = if (action.exec.pull) blackboxInvokers(invokers)
else managedInvokers(invokers)
val invokersWithUsage = invokersToUse.view.map {
// Using a view defers the comparably expensive lookup to actual
access of the element
- case invoker => (invoker.id, invoker.status,
currentActivations.getOrElse(instance.toString, 0))
+ case invoker => (invoker.id, invoker.status,
currentActivations.getOrElse(invoker.id.toString, 0))
Review comment:
This line was the issue.
Instead of the invoker-instance, the controller instance has been used.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cbickel opened a new pull request #3221: Fix overlaod of invoker.
cbickel opened a new pull request #3221: Fix overlaod of invoker. URL: https://github.com/apache/incubator-openwhisk/pull/3221 Due to duplicate names of variables, the loadbalancer thought, that the home invoker of an action has still some free slots. But the controller asked for the wrong invoker, so the invoker was totally overloaded. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jthomas commented on issue #22: Added local installation documentation link to openwhisk-devtools/docker-compose docs
jthomas commented on issue #22: Added local installation documentation link to openwhisk-devtools/docker-compose docs URL: https://github.com/apache/incubator-openwhisk-workshop/pull/22#issuecomment-360772270 LGTM. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jthomas commented on issue #91: Add documentation for adding the OpenWhisk catalog
jthomas commented on issue #91: Add documentation for adding the OpenWhisk catalog URL: https://github.com/apache/incubator-openwhisk-devtools/pull/91#issuecomment-360772115 I've just updated the catalog PR to include docs now: https://github.com/apache/incubator-openwhisk-devtools/pull/83/files This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mbehrendt commented on issue #3220: Move log message into a closure to defer string interpolation.
mbehrendt commented on issue #3220: Move log message into a closure to defer string interpolation. URL: https://github.com/apache/incubator-openwhisk/pull/3220#issuecomment-360737034 excellent. @cbickel can you pls post the absolute numbers of your measurements? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mbehrendt commented on issue #3220: Move log message into a closure to defer string interpolation.
mbehrendt commented on issue #3220: Move log message into a closure to defer string interpolation. URL: https://github.com/apache/incubator-openwhisk/pull/3220#issuecomment-360737034 excellent. @cbickel can you pls post the absolute numbers of your measurements? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes closed pull request #3220: Move log message into a closure to defer string interpolation.
markusthoemmes closed pull request #3220: Move log message into a closure to
defer string interpolation.
URL: https://github.com/apache/incubator-openwhisk/pull/3220
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/common/scala/src/main/scala/whisk/common/Logging.scala
b/common/scala/src/main/scala/whisk/common/Logging.scala
index 873bcef81b..5a18ee3468 100644
--- a/common/scala/src/main/scala/whisk/common/Logging.scala
+++ b/common/scala/src/main/scala/whisk/common/Logging.scala
@@ -34,9 +34,9 @@ trait Logging {
* Prints a message on DEBUG level
*
* @param from Reference, where the method was called from.
- * @param message Message to write to the log
+ * @param message Message to write to the log if not empty
*/
- def debug(from: AnyRef, message: String)(implicit id: TransactionId =
TransactionId.unknown) = {
+ def debug(from: AnyRef, message: => String)(implicit id: TransactionId =
TransactionId.unknown) = {
if (id.meta.extraLogging) {
emit(InfoLevel, id, from, message)
} else {
@@ -48,9 +48,9 @@ trait Logging {
* Prints a message on INFO level
*
* @param from Reference, where the method was called from.
- * @param message Message to write to the log
+ * @param message Message to write to the log if not empty
*/
- def info(from: AnyRef, message: String)(implicit id: TransactionId =
TransactionId.unknown) = {
+ def info(from: AnyRef, message: => String)(implicit id: TransactionId =
TransactionId.unknown) = {
emit(InfoLevel, id, from, message)
}
@@ -58,9 +58,9 @@ trait Logging {
* Prints a message on WARN level
*
* @param from Reference, where the method was called from.
- * @param message Message to write to the log
+ * @param message Message to write to the log if not empty
*/
- def warn(from: AnyRef, message: String)(implicit id: TransactionId =
TransactionId.unknown) = {
+ def warn(from: AnyRef, message: => String)(implicit id: TransactionId =
TransactionId.unknown) = {
emit(WarningLevel, id, from, message)
}
@@ -68,9 +68,9 @@ trait Logging {
* Prints a message on ERROR level
*
* @param from Reference, where the method was called from.
- * @param message Message to write to the log
+ * @param message Message to write to the log if not empty
*/
- def error(from: AnyRef, message: String)(implicit id: TransactionId =
TransactionId.unknown) = {
+ def error(from: AnyRef, message: => String)(implicit id: TransactionId =
TransactionId.unknown) = {
emit(ErrorLevel, id, from, message)
}
@@ -80,19 +80,22 @@ trait Logging {
* @param loglevel The level to log on
* @param id TransactionId to include in the log
* @param from Reference, where the method was called from.
- * @param message Message to write to the log
+ * @param message Message to write to the log if not empty
*/
- def emit(loglevel: LogLevel, id: TransactionId, from: AnyRef, message:
String)
+ protected[common] def emit(loglevel: LogLevel, id: TransactionId, from:
AnyRef, message: => String)
}
/**
* Implementation of Logging, that uses Akka logging.
*/
class AkkaLogging(loggingAdapter: LoggingAdapter) extends Logging {
- def emit(loglevel: LogLevel, id: TransactionId, from: AnyRef, message:
String) = {
+ def emit(loglevel: LogLevel, id: TransactionId, from: AnyRef, message: =>
String) = {
if (loggingAdapter.isEnabled(loglevel)) {
- val name = if (from.isInstanceOf[String]) from else
Logging.getCleanSimpleClassName(from.getClass)
- loggingAdapter.log(loglevel, s"[$id] [$name] $message")
+ val logmsg: String = message // generates the message
+ if (logmsg.nonEmpty) { // log it only if its not empty
+val name = if (from.isInstanceOf[String]) from else
Logging.getCleanSimpleClassName(from.getClass)
+loggingAdapter.log(loglevel, s"[$id] [$name] $logmsg")
+ }
}
}
}
@@ -101,7 +104,7 @@ class AkkaLogging(loggingAdapter: LoggingAdapter) extends
Logging {
* Implementaion of Logging, that uses the output stream.
*/
class PrintStreamLogging(outputStream: PrintStream = Console.out) extends
Logging {
- def emit(loglevel: LogLevel, id: TransactionId, from: AnyRef, message:
String) = {
+ override def emit(loglevel: LogLevel, id: TransactionId, from: AnyRef,
message: => String) = {
val now = Instant.now(Clock.systemUTC)
val time = Emitter.timeFormat.format(now)
val name = if (from.isInstanceOf[String]) from else
Logging.getCleanSimpleClassName(from.getClass)
diff --git a/common/scala/src/main/scala/whisk/common/TransactionId.scala
b/common/scala/src/main/scala/whisk/common/TransactionId.scala
index 72dde1e05f..09ca1fd894 100644
---
[GitHub] cbickel commented on issue #3220: Move log message into a closure to defer string interpolation.
cbickel commented on issue #3220: Move log message into a closure to defer string interpolation. URL: https://github.com/apache/incubator-openwhisk/pull/3220#issuecomment-360736219 ? Great PR. I did some load measurements and saw an increase of trigger get of 11%. In both versions, I set `metrics.log.enabled` to false. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cbickel commented on issue #3220: Move log message into a closure to defer string interpolation.
cbickel commented on issue #3220: Move log message into a closure to defer string interpolation. URL: https://github.com/apache/incubator-openwhisk/pull/3220#issuecomment-360736219 ? Great PR. I did some load measurements and saw an increase of trigger get of 11%. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
