[GitHub] upgle commented on issue #3880: Modify that web action in the bound package can be accessed.
upgle commented on issue #3880: Modify that web action in the bound package can be accessed. URL: https://github.com/apache/incubator-openwhisk/pull/3880#issuecomment-405807606 @rabbah I modified it to check the entitlement and added tests. Now, It returns the authentication error if a web action has no entitlement. ``` { "error": "The supplied authentication is not authorized to access 'whisk.system/share'.", "code": "NmSIY4dJanMT7YS3MPwFoOPL4Np72g5B" } ``` Could you please remove WIP label and review again? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3880: Modify that web action in the bound package can be accessed.
codecov-io edited a comment on issue #3880: Modify that web action in the bound package can be accessed. URL: https://github.com/apache/incubator-openwhisk/pull/3880#issuecomment-405214016 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=h1) Report > Merging [#3880](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `4.76%`. > The diff coverage is `88.88%`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/graphs/tree.svg?src=pr=l0YmsiSAso=650=150)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3880 +/- ## == - Coverage 75.78% 71.02% -4.77% == Files 145 145 Lines6897 6902 +5 Branches 418 423 +5 == - Hits 5227 4902 -325 - Misses 1670 2000 +330 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=tree) | Coverage Δ | | |---|---|---| | [.../main/scala/whisk/core/controller/WebActions.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udHJvbGxlci9XZWJBY3Rpb25zLnNjYWxh) | `88.37% <88.88%> (-0.17%)` | :arrow_down: | | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0% <0%> (-95.08%)` | :arrow_down: | | [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0% <0%> (-92.6%)` | :arrow_down: | | [...whisk/core/database/cosmosdb/CosmosDBSupport.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJTdXBwb3J0LnNjYWxh) | `0% <0%> (-81.82%)` | :arrow_down: | | [...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlUHJvdmlkZXIuc2NhbGE=) | `0% <0%> (-58.83%)` | :arrow_down: | | [...la/whisk/core/database/cosmosdb/CosmosDBUtil.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJVdGlsLnNjYWxh) | `92% <0%> (-4%)` | :arrow_down: | | [...cala/src/main/scala/whisk/http/ErrorResponse.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2h0dHAvRXJyb3JSZXNwb25zZS5zY2FsYQ==) | `90.9% <0%> (+1.13%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=footer). Last update [e96c1bb...5686375](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3880: Modify that web action in the bound package can be accessed.
codecov-io edited a comment on issue #3880: Modify that web action in the bound package can be accessed. URL: https://github.com/apache/incubator-openwhisk/pull/3880#issuecomment-405214016 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=h1) Report > Merging [#3880](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `5.57%`. > The diff coverage is `88.88%`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/graphs/tree.svg?width=650=pr=150=l0YmsiSAso)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3880 +/- ## == - Coverage 75.78% 70.21% -5.58% == Files 145 145 Lines6897 6902 +5 Branches 418 423 +5 == - Hits 5227 4846 -381 - Misses 1670 2056 +386 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=tree) | Coverage Δ | | |---|---|---| | [.../main/scala/whisk/core/controller/WebActions.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udHJvbGxlci9XZWJBY3Rpb25zLnNjYWxh) | `88.37% <88.88%> (-0.17%)` | :arrow_down: | | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0% <0%> (-95.08%)` | :arrow_down: | | [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0% <0%> (-92.6%)` | :arrow_down: | | [...whisk/core/database/cosmosdb/CosmosDBSupport.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJTdXBwb3J0LnNjYWxh) | `0% <0%> (-81.82%)` | :arrow_down: | | [...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlUHJvdmlkZXIuc2NhbGE=) | `0% <0%> (-58.83%)` | :arrow_down: | | [...whisk/connector/kafka/KafkaMessagingProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2Nvbm5lY3Rvci9rYWZrYS9LYWZrYU1lc3NhZ2luZ1Byb3ZpZGVyLnNjYWxh) | `23.07% <0%> (-46.16%)` | :arrow_down: | | [...whisk/connector/kafka/KafkaProducerConnector.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2Nvbm5lY3Rvci9rYWZrYS9LYWZrYVByb2R1Y2VyQ29ubmVjdG9yLnNjYWxh) | `22.22% <0%> (-36.12%)` | :arrow_down: | | [...whisk/connector/kafka/KafkaConsumerConnector.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2Nvbm5lY3Rvci9rYWZrYS9LYWZrYUNvbnN1bWVyQ29ubmVjdG9yLnNjYWxh) | `33.33% <0%> (-29.63%)` | :arrow_down: | | [...rc/main/scala/whisk/core/entity/ExecManifest.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZW50aXR5L0V4ZWNNYW5pZmVzdC5zY2FsYQ==) | `81.48% <0%> (-14.82%)` | :arrow_down: | | ... and [4 more](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3880?src=pr=footer). Last update
[GitHub] jiangpengcheng commented on issue #3853: Set erlang magic cookie for couchdb
jiangpengcheng commented on issue #3853: Set erlang magic cookie for couchdb URL: https://github.com/apache/incubator-openwhisk/pull/3853#issuecomment-405773126 @vvraskin I use the `local_action` and `run_once` options, I think it does the same thing This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3812: ContainerClient + akka http alternative to HttpUtils
codecov-io edited a comment on issue #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#issuecomment-401212372 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=h1) Report > Merging [#3812](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `4.59%`. > The diff coverage is `57.89%`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/graphs/tree.svg?src=pr=l0YmsiSAso=650=150)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=tree) ```diff @@Coverage Diff@@ ## master#3812 +/- ## = - Coverage 75.78% 71.19% -4.6% = Files 145 146 +1 Lines6897 6943 +46 Branches 418 435 +17 = - Hits 5227 4943-284 - Misses 1670 2000+330 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=tree) | Coverage Δ | | |---|---|---| | [...la/whisk/core/containerpool/ContainerFactory.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJGYWN0b3J5LnNjYWxh) | `100% <ø> (ø)` | :arrow_up: | | [...la/src/main/scala/whisk/core/mesos/MesosTask.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvbWVzb3MvTWVzb3NUYXNrLnNjYWxh) | `86.11% <ø> (ø)` | :arrow_up: | | [...containerpool/kubernetes/KubernetesContainer.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9rdWJlcm5ldGVzL0t1YmVybmV0ZXNDb250YWluZXIuc2NhbGE=) | `91.66% <ø> (ø)` | :arrow_up: | | [...sk/core/containerpool/docker/DockerContainer.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9kb2NrZXIvRG9ja2VyQ29udGFpbmVyLnNjYWxh) | `79.16% <0%> (+2.13%)` | :arrow_up: | | [...n/scala/whisk/core/database/CouchDbRestStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvQ291Y2hEYlJlc3RTdG9yZS5zY2FsYQ==) | `73.23% <100%> (ø)` | :arrow_up: | | [.../src/main/scala/whisk/http/PoolingRestClient.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2h0dHAvUG9vbGluZ1Jlc3RDbGllbnQuc2NhbGE=) | `90.9% <100%> (+1.62%)` | :arrow_up: | | [...ain/scala/whisk/core/containerpool/HttpUtils.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9IdHRwVXRpbHMuc2NhbGE=) | `69.81% <14.28%> (+18.9%)` | :arrow_up: | | [...ain/scala/whisk/core/containerpool/Container.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXIuc2NhbGE=) | `76.56% <25%> (+0.37%)` | :arrow_up: | | [...ala/whisk/core/containerpool/ContainerClient.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJDbGllbnQuc2NhbGE=) | `70.45% <70.45%> (ø)` | | | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | ... and [7 more](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=footer). Last update [e96c1bb...952cde3](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[GitHub] tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r203219670 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() Review comment: @markusthoemmes WDYT? This may affect use of `AutoCloseable` - for now I will return a Unit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r203219371 ## File path: common/scala/src/main/scala/whisk/http/PoolingRestClient.scala ## @@ -43,29 +45,36 @@ class PoolingRestClient( host: String, port: Int, queueSize: Int, - httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None)( - implicit system: ActorSystem) { + httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None, + timeout: Option[FiniteDuration] = None)(implicit system: ActorSystem) { require(protocol == "http" || protocol == "https", "Protocol must be one of { http, https }.") protected implicit val context: ExecutionContext = system.dispatcher protected implicit val materializer: ActorMaterializer = ActorMaterializer() + //if specified, override the ClientConnection idle-timeout value + private val timeoutSettings = +ConnectionPoolSettings(system.settings.config) + .withConnectionSettings(if (timeout.isDefined) { +ClientConnectionSettings(system.settings.config) + .withIdleTimeout(timeout.get) + } else { ClientConnectionSettings(system.settings.config) }) Review comment: nice This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r203218558 ## File path: common/scala/src/main/scala/whisk/core/database/CouchDbRestStore.scala ## @@ -513,7 +512,7 @@ class CouchDbRestStore[DocumentAbstraction <: DocumentSerializer](dbProtocol: St .getOrElse(Future.successful(true)) // For CouchDB it is expected that the entire document is deleted. override def shutdown(): Unit = { -Await.ready(client.shutdown(), 1.minute) +client.shutdown() Review comment: Yes! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r203218116 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r203217847 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
tysonnorris commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r203217369 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] pritidesai closed pull request #223: Update the release folder on a repository basis
pritidesai closed pull request #223: Update the release folder on a repository basis URL: https://github.com/apache/incubator-openwhisk-release/pull/223 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/releases/0.9.0-incubating/INSTALL.md b/releases/0.9.0-incubating/INSTALL.md index f4415e4..39d2be0 100644 --- a/releases/0.9.0-incubating/INSTALL.md +++ b/releases/0.9.0-incubating/INSTALL.md @@ -19,11 +19,11 @@ # Download OpenWhisk -The OpenWhisk source code may be downloaded from https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc2. The current release is version 0.9.0, and the artifact for this OpenWhisk source code release is called `"openwhisk-0.9.0-incubating-sources.tar.gz"`. +The OpenWhisk source code may be downloaded from https://dist.apache.org/repos/dist/release/incubator/openwhisk/apache-openwhisk-0.9.0-incubating. The current release is version 0.9.0, and the artifact for this OpenWhisk source code release is called `"openwhisk-0.9.0-incubating-sources.tar.gz"`. ``` # download from your terminal with wget -wget https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc2/openwhisk-0.9.0-incubating-sources.tar.gz +wget https://dist.apache.org/repos/dist/release/incubator/openwhisk/apache-openwhisk-0.9.0-incubating/openwhisk-0.9.0-incubating-sources.tar.gz ``` # Verify the SHA-512 checksums, and signature @@ -42,7 +42,7 @@ You need to install `gpg` on your local machine. Before using `gpg` to verify the OpenWhisk release integrity, you should [verify `gpg`'s own integrity](https://gnupg.org/download/integrity_check.html). -The public key used to verify the OpenWhisk checksums can be found [here](https://dist.apache.org/repos/dist/dev/incubator/openwhisk/KEYS). Download the key and import it on your local machine. +The public key used to verify the OpenWhisk checksums can be found [here](https://dist.apache.org/repos/dist/release/incubator/openwhisk/KEYS). Download the key and import it on your local machine. ``` gpg --import @@ -56,9 +56,9 @@ To generate the SHA512 checksum: gpg --print-md SHA512 ``` -The parameter is the file of the artifact `"openwhisk-0.9.0-incubating-sources.tar.gz"`. Compare the content with the [SHA512 file](https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc2/openwhisk-0.9.0-incubating-sources.tar.gz.sha512). +The parameter is the file of the artifact `"openwhisk-0.9.0-incubating-sources.tar.gz"`. Compare the content with the [SHA512 file](https://dist.apache.org/repos/dist/release/incubator/openwhisk/apache-openwhisk-0.9.0-incubating/openwhisk-0.9.0-incubating-sources.tar.gz.sha512). -Download the [signature](https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc2/openwhisk-0.9.0-incubating-sources.tar.gz.asc), and verify it with the command: +Download the [signature](https://dist.apache.org/repos/dist/release/incubator/openwhisk/apache-openwhisk-0.9.0-incubating/openwhisk-0.9.0-incubating-sources.tar.gz.asc), and verify it with the command: ``` gpg --verify openwhisk-0.9.0-incubating-sources.tar.gz.asc openwhisk-0.9.0-incubating-sources.tar.gz diff --git a/tools/move_stage_to_release.sh b/tools/move_stage_to_release.sh index 6a81e40..abc2d3e 100755 --- a/tools/move_stage_to_release.sh +++ b/tools/move_stage_to_release.sh @@ -26,13 +26,30 @@ SVN_PASSWORD=$2 source "$SCRIPTDIR/load_config.sh" "$SVN_USERNAME" "$SVN_PASSWORD" -if [[ `wget -S --spider $RELEASE_VERSION_URL 2>&1 | grep 'HTTP/1.1 200 OK'` ]]; then -svn delete $RELEASE_VERSION_URL -m "Removing Apache OpenWhisk release ${full_version}." $CREDENTIALS +if [[ `wget -S --spider $RELEASE_VERSION_URL 2>&1 | grep 'HTTP/1.1 404 Not Found'` ]]; then +svn mkdir $RELEASE_VERSION_URL -m "Creating Apache OpenWhisk release ${version}." $CREDENTIALS fi -if [[ `wget -S --spider $CURRENT_VERSION_URL 2>&1 | grep 'HTTP/1.1 200 OK'` ]]; then -svn copy $CURRENT_VERSION_URL $RELEASE_URL -m "Releasing Apache OpenWhisk release ${full_version}." $CREDENTIALS -if [ "$full_version" != "$version" ]; then -svn mv $RELEASE_VERSION_URL_STAGE $RELEASE_VERSION_URL -m "Remaning the directory from ${full_version} to ${version}." $CREDENTIALS -fi -fi +# Create a subversion directory for openwhisk to stage all the packages +rm -rf $OPENWHISK_SVN +mkdir -p $OPENWHISK_SVN +cd $OPENWHISK_SVN + +# Check out the artifacts in the staging URL to a local directory. +svn co $CURRENT_VERSION_URL $REMOTE_PATH + +# Check out the release directory to a local directory. +svn co $RELEASE_VERSION_URL $REMOTE_PATH_RELEASE + +for repo in $(echo $repos | sed "s/,/
[GitHub] eweiter opened a new pull request #73: Don't override runtime npm packages when user provides their own
eweiter opened a new pull request #73: Don't override runtime npm packages when user provides their own URL: https://github.com/apache/incubator-openwhisk-runtime-nodejs/pull/73 See #58 When a user inherits from our runtimes and they run an npm install with their own package.json it will clobber our npm modules. The solution was to move the runtime's npm modules to the root of the Docker container so that when a user installs their own they will be at the user level of the container and if the runtime cannot find the npm module in the user level it will default up to trying to find it from the root level. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3799: Display proper error when sequence invocation fails due to missing component
rabbah commented on issue #3799: Display proper error when sequence invocation fails due to missing component URL: https://github.com/apache/incubator-openwhisk/pull/3799#issuecomment-405693390 Thanks @markusthoemmes. That's much simpler and preserves the pipelining between fetches and invokes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes commented on issue #3799: Display proper error when sequence invocation fails due to missing component
markusthoemmes commented on issue #3799: Display proper error when sequence invocation fails due to missing component URL: https://github.com/apache/incubator-openwhisk/pull/3799#issuecomment-405692374 I think this fix has moved a bit in the wrong direction. The mode @rabbah wants to keep is lazy evaluation of the Futures. While it is true, that all actions will be fetched at the same time, invocation will only wait for the first Future to complete before it starts invoking the sequence. The failing operation will be `invokeNextAction`. We can either attach `recoverWith` to the `invokeNextAction` call or even recover this within `invokeNextAction`. We can then use the current `accounting` object (since that's known when calling `invokeNextAction`) to apply the failure there. That should keep state up to that point (another thing mentioned by @rabbah) and should keep the change simple. Thoughts? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component
rabbah commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component URL: https://github.com/apache/incubator-openwhisk/pull/3799#discussion_r203134728 ## File path: core/controller/src/main/scala/whisk/core/controller/actions/SequenceActions.scala ## @@ -247,40 +245,47 @@ protected[actions] trait SequenceActions { // // This action/parameter resolution is done in futures; the execution starts as soon as the first component // is resolved. -val resolvedFutureActions = resolveDefaultNamespace(components, user) map { c => +val resolvedFutureActions: Seq[Future[WhiskActionMetaData]] = resolveDefaultNamespace(components, user) map { c => WhiskActionMetaData.resolveActionAndMergeParameters(entityStore, c) } // this holds the initial value of the accounting structure, including the input boxed as an ActivationResponse -val initialAccounting = Future.successful { - SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) -} - -// execute the actions in sequential blocking fashion -resolvedFutureActions - .foldLeft(initialAccounting) { (accountingFuture, futureAction) => -accountingFuture.flatMap { accounting => - if (accounting.atomicActionCnt < actionSequenceLimit) { -invokeNextAction(user, futureAction, accounting, cause).flatMap { accounting => - if (!accounting.shortcircuit) { -Future.successful(accounting) - } else { -// this is to short circuit the fold -Future.failed(FailedSequenceActivation(accounting)) // terminates the fold - } +val initialAccounting = SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) + +Future + .sequence(resolvedFutureActions) + .flatMap(actions => Review comment: i think this is safer but now we wait for all the futures to resolve on the fetch - and don't pipeline any of the invokes, so worse on performance for a long sequence. i think we need 1. additional test to cover missing package 2. additional test to cover missing bindings 3. a test with 2 sequence components, where the first is fetched but the second fails, the accounting object should show the execution that accounts for the first component's execution. 1 & 2 to make sure all no doc exceptions are handles consistently. 3 will require refactoring the code in the function so that you can test the independent components without having to mess with the data store. This is what I meant by refactoring. in other words im saying if you have a long sequence `a,b,c,d,e` you might execute some of the sequence say `a,b,c` before you realize d failed (no doc exception); the recovery that you added uses the accounting from time = 0 so you'd lose the activations fo `a,b,c`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component
rabbah commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component URL: https://github.com/apache/incubator-openwhisk/pull/3799#discussion_r203134728 ## File path: core/controller/src/main/scala/whisk/core/controller/actions/SequenceActions.scala ## @@ -247,40 +245,47 @@ protected[actions] trait SequenceActions { // // This action/parameter resolution is done in futures; the execution starts as soon as the first component // is resolved. -val resolvedFutureActions = resolveDefaultNamespace(components, user) map { c => +val resolvedFutureActions: Seq[Future[WhiskActionMetaData]] = resolveDefaultNamespace(components, user) map { c => WhiskActionMetaData.resolveActionAndMergeParameters(entityStore, c) } // this holds the initial value of the accounting structure, including the input boxed as an ActivationResponse -val initialAccounting = Future.successful { - SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) -} - -// execute the actions in sequential blocking fashion -resolvedFutureActions - .foldLeft(initialAccounting) { (accountingFuture, futureAction) => -accountingFuture.flatMap { accounting => - if (accounting.atomicActionCnt < actionSequenceLimit) { -invokeNextAction(user, futureAction, accounting, cause).flatMap { accounting => - if (!accounting.shortcircuit) { -Future.successful(accounting) - } else { -// this is to short circuit the fold -Future.failed(FailedSequenceActivation(accounting)) // terminates the fold - } +val initialAccounting = SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) + +Future + .sequence(resolvedFutureActions) + .flatMap(actions => Review comment: i think this is safer but now we wait for all the futures to resolve on the fetch - and don't pipeline any of the invokes, so worse on performance for a long sequence. i think we need 1. additional test to cover missing package 2. additional test to cover missing bindings 3. a test with 2 sequence components, where the first is fetched but the second fails, the accounting object should show the execution that accounts for the first component's execution. 1 & 2 to make sure all no doc exceptions are handles consistently. 3 will require refactoring the code in the function so that you can test the independent components without having to mess with the data store. This is what I meant by refactoring. in other words im saying if you have a long sequence `a,b,c,d,e` you might execute some of the sequence say `a,b,c` before you realize `d` failed (no doc exception); the recovery that you added uses the accounting from time = 0 so you'd lose the activations fo `a,b,c`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component
rabbah commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component URL: https://github.com/apache/incubator-openwhisk/pull/3799#discussion_r203134728 ## File path: core/controller/src/main/scala/whisk/core/controller/actions/SequenceActions.scala ## @@ -247,40 +245,47 @@ protected[actions] trait SequenceActions { // // This action/parameter resolution is done in futures; the execution starts as soon as the first component // is resolved. -val resolvedFutureActions = resolveDefaultNamespace(components, user) map { c => +val resolvedFutureActions: Seq[Future[WhiskActionMetaData]] = resolveDefaultNamespace(components, user) map { c => WhiskActionMetaData.resolveActionAndMergeParameters(entityStore, c) } // this holds the initial value of the accounting structure, including the input boxed as an ActivationResponse -val initialAccounting = Future.successful { - SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) -} - -// execute the actions in sequential blocking fashion -resolvedFutureActions - .foldLeft(initialAccounting) { (accountingFuture, futureAction) => -accountingFuture.flatMap { accounting => - if (accounting.atomicActionCnt < actionSequenceLimit) { -invokeNextAction(user, futureAction, accounting, cause).flatMap { accounting => - if (!accounting.shortcircuit) { -Future.successful(accounting) - } else { -// this is to short circuit the fold -Future.failed(FailedSequenceActivation(accounting)) // terminates the fold - } +val initialAccounting = SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) + +Future + .sequence(resolvedFutureActions) + .flatMap(actions => Review comment: i think this is safer but now we wait for all the futures to resolve on the fetch - and don't pipeline any of the invokes, so worse on performance for a long sequence. i think we need 1. additional test to cover missing package 2. additional test to cover missing bindings 3. a test with 2 sequence components, where the first is fetched but the second fails, the accounting object should show the execution that accounts for the first component's execution. 1 & 2 to make sure all no doc exceptions are handles consistently. 3 will require refactoring the code in the function so that you can test the independent components without having to mess with the data store. This is what I meant by refactoring. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] StrayCat1 commented on issue #257: External zookeeper kafka
StrayCat1 commented on issue #257: External zookeeper kafka URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/257#issuecomment-405661256 Thanks @dgrove-oss , I just submitted the CLA and Ill let you know when I hear back. Also, I tested those changes to _helpers.tpl and realized I forgot to add the same toggle for zookeeper_zero_host. It does work though. I was able to deploy my kafka and zookeeper charts first, then deploy the openwhisk chart with its kafka and zookeeper toggled off and the name of the existing kafka and zookeeper services configured as mentioned above. I also updated the docs with an example. Ill need to update my pull request to include those changes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3884: Document metrics generated within OpenWhisk
codecov-io commented on issue #3884: Document metrics generated within OpenWhisk URL: https://github.com/apache/incubator-openwhisk/pull/3884#issuecomment-405653420 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884?src=pr=h1) Report > Merging [#3884](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `4.78%`. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884/graphs/tree.svg?height=150=650=l0YmsiSAso=pr)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884?src=pr=tree) ```diff @@Coverage Diff@@ ## master #3884 +/- ## = - Coverage 75.78% 71% -4.79% = Files 145 145 Lines68976897 Branches 418 418 = - Hits 52274897 -330 - Misses 16702000 +330 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884?src=pr=tree) | Coverage Δ | | |---|---|---| | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0% <0%> (-95.08%)` | :arrow_down: | | [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0% <0%> (-92.6%)` | :arrow_down: | | [...whisk/core/database/cosmosdb/CosmosDBSupport.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJTdXBwb3J0LnNjYWxh) | `0% <0%> (-81.82%)` | :arrow_down: | | [...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlUHJvdmlkZXIuc2NhbGE=) | `0% <0%> (-58.83%)` | :arrow_down: | | [...la/whisk/core/database/cosmosdb/CosmosDBUtil.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJVdGlsLnNjYWxh) | `92% <0%> (-4%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884?src=pr=footer). Last update [e96c1bb...4b8d3ba](https://codecov.io/gh/apache/incubator-openwhisk/pull/3884?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah edited a comment on issue #3883: Make it easier to use webpack with JS actions
rabbah edited a comment on issue #3883: Make it easier to use webpack with JS actions URL: https://github.com/apache/incubator-openwhisk/issues/3883#issuecomment-405638973 no worries. We can move the issue if necessary. I think it’s worth having the discussion here first where there are more This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3883: Make it easier to use webpack with JS actions
rabbah commented on issue #3883: Make it easier to use webpack with JS actions URL: https://github.com/apache/incubator-openwhisk/issues/3883#issuecomment-405638973 no worries. We can move the issue of necessary. I think it’s worth having the discussion here first where there are more This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] remore commented on issue #3725: Add Ruby2.5 runtime support
remore commented on issue #3725: Add Ruby2.5 runtime support URL: https://github.com/apache/incubator-openwhisk/pull/3725#issuecomment-405635657 The first PR has been sent: https://github.com/apache/incubator-openwhisk-runtime-ruby/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] remore opened a new pull request #1: Add the base source code for Ruby 2.5 Runtime
remore opened a new pull request #1: Add the base source code for Ruby 2.5 Runtime URL: https://github.com/apache/incubator-openwhisk-runtime-ruby/pull/1 Related Issue(s): https://github.com/apache/incubator-openwhisk/pull/3725 https://github.com/remore/openwhisk-runtime-ruby/issues?q=is%3Aissue+is%3Aclosed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] macdonst commented on issue #3883: Make it easier to use webpack with JS actions
macdonst commented on issue #3883: Make it easier to use webpack with JS actions URL: https://github.com/apache/incubator-openwhisk/issues/3883#issuecomment-405633048 @rabbah If I've open the issue in the wrong place than yeah, I'm more than happy to move this issue to another repo. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cduque89 commented on issue #258: Where should the come from for GKE nginx ingress?
cduque89 commented on issue #258: Where should the come from for GKE nginx ingress? URL: https://github.com/apache/incubator-openwhisk-deploy-kube/issues/258#issuecomment-405621290 Hi @nerdguru. If that's the case you can still have your DNS domain as described in the documentation link and use it in different K8s clusters. Or you might decide to configure it just with a static external-ip as described in the previous section of the docs: https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer maybe it suites you best. We never used this approach since we find it easier with the first one. Not that even with the situation of bringing K8s clusters up and down frequently you can keep the same gcloud domains since it's an independent resource in your gcloud account. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #97: remove references to bluemix
rabbah commented on issue #97: remove references to bluemix URL: https://github.com/apache/incubator-openwhisk-client-go/issues/97#issuecomment-405620225 yes i think we should remove them. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubee commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component
dubee commented on a change in pull request #3799: Display proper error when sequence invocation fails due to missing component URL: https://github.com/apache/incubator-openwhisk/pull/3799#discussion_r203062593 ## File path: core/controller/src/main/scala/whisk/core/controller/actions/SequenceActions.scala ## @@ -247,40 +245,47 @@ protected[actions] trait SequenceActions { // // This action/parameter resolution is done in futures; the execution starts as soon as the first component // is resolved. -val resolvedFutureActions = resolveDefaultNamespace(components, user) map { c => +val resolvedFutureActions: Seq[Future[WhiskActionMetaData]] = resolveDefaultNamespace(components, user) map { c => WhiskActionMetaData.resolveActionAndMergeParameters(entityStore, c) } // this holds the initial value of the accounting structure, including the input boxed as an ActivationResponse -val initialAccounting = Future.successful { - SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) -} - -// execute the actions in sequential blocking fashion -resolvedFutureActions - .foldLeft(initialAccounting) { (accountingFuture, futureAction) => -accountingFuture.flatMap { accounting => - if (accounting.atomicActionCnt < actionSequenceLimit) { -invokeNextAction(user, futureAction, accounting, cause).flatMap { accounting => - if (!accounting.shortcircuit) { -Future.successful(accounting) - } else { -// this is to short circuit the fold -Future.failed(FailedSequenceActivation(accounting)) // terminates the fold - } +val initialAccounting = SequenceAccounting(atomicActionCnt, ActivationResponse.payloadPlaceholder(inputPayload)) + +Future + .sequence(resolvedFutureActions) + .flatMap(actions => Review comment: @rabbah, what do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] drcariel commented on issue #97: remove references to bluemix
drcariel commented on issue #97: remove references to bluemix URL: https://github.com/apache/incubator-openwhisk-client-go/issues/97#issuecomment-405616640 @rabbah I see these bluemix references in cli and openwhisk repos too - should those be removed as well? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nerdguru commented on issue #258: Where should the come from for GKE nginx ingress?
nerdguru commented on issue #258: Where should the come from for GKE nginx ingress? URL: https://github.com/apache/incubator-openwhisk-deploy-kube/issues/258#issuecomment-405605697 Hi Carlos and thanks for the quick reply. So does that imply that a domain name for my GKE cluster is required for OpenWhisk? For development situations where I may be bringing K8S clusters up and down frequently, being able to use an IP address would be more convenient. Or am I missing something? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cduque89 opened a new pull request #259: ingress.md information about google cloud DNS
cduque89 opened a new pull request #259: ingress.md information about google cloud DNS URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/259 As stated in #258 it was not evident how to get a DNS name for the google cloud. I've updated the documentation with a link to the official Google Cloud docs. Hope this helps in the future. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3779: S3AttachmentStore
codecov-io edited a comment on issue #3779: S3AttachmentStore URL: https://github.com/apache/incubator-openwhisk/pull/3779#issuecomment-398322147 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779?src=pr=h1) Report > Merging [#3779](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `4.68%`. > The diff coverage is `84.48%`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/graphs/tree.svg?width=650=150=pr=l0YmsiSAso)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779?src=pr=tree) ```diff @@Coverage Diff@@ ## master #3779 +/- ## = - Coverage 75.78% 71.1% -4.69% = Files 145 146 +1 Lines68976953 +56 Branches 418 418 = - Hits 52274944 -283 - Misses 16702009 +339 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779?src=pr=tree) | Coverage Δ | | |---|---|---| | [...n/scala/whisk/core/database/CouchDbRestStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvQ291Y2hEYlJlc3RTdG9yZS5zY2FsYQ==) | `73.23% <0%> (ø)` | :arrow_up: | | [.../scala/whisk/core/database/AttachmentSupport.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvQXR0YWNobWVudFN1cHBvcnQuc2NhbGE=) | `100% <100%> (ø)` | :arrow_up: | | [.../scala/src/main/scala/whisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `92.06% <100%> (+0.06%)` | :arrow_up: | | [...ala/whisk/core/database/s3/S3AttachmentStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvczMvUzNBdHRhY2htZW50U3RvcmUuc2NhbGE=) | `85.18% <85.18%> (ø)` | | | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0% <0%> (-95.08%)` | :arrow_down: | | [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0% <0%> (-92.6%)` | :arrow_down: | | [...whisk/core/database/cosmosdb/CosmosDBSupport.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJTdXBwb3J0LnNjYWxh) | `0% <0%> (-81.82%)` | :arrow_down: | | [...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlUHJvdmlkZXIuc2NhbGE=) | `0% <0%> (-58.83%)` | :arrow_down: | | [...la/whisk/core/database/cosmosdb/CosmosDBUtil.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJVdGlsLnNjYWxh) | `92% <0%> (-4%)` | :arrow_down: | | ... and [2 more](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779?src=pr=footer). Last update [e96c1bb...5966452](https://codecov.io/gh/apache/incubator-openwhisk/pull/3779?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[GitHub] dgrove-oss commented on issue #257: External zookeeper kafka
dgrove-oss commented on issue #257: External zookeeper kafka URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/257#issuecomment-405587098 Hi @StrayCat1, thanks for the contribution! If you haven't done so already, could you please sign an Apache CLA (see https://github.com/apache/incubator-openwhisk/blob/master/CONTRIBUTING.md)? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cduque89 commented on issue #258: Where should the come from for GKE nginx ingress?
cduque89 commented on issue #258: Where should the come from for GKE nginx ingress? URL: https://github.com/apache/incubator-openwhisk-deploy-kube/issues/258#issuecomment-405582272 Hi @nerdguru. You are the one defining the domain in your google cloud. Please give a look into: https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip Hope this helps. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3883: Make it easier to use webpack with JS actions
rabbah commented on issue #3883: Make it easier to use webpack with JS actions URL: https://github.com/apache/incubator-openwhisk/issues/3883#issuecomment-405580411 @eweiter @csantanapr I think it would make sense to support this in the node runtime - should we move the issue there? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3884: Document metrics generated within OpenWhisk
rabbah commented on issue #3884: Document metrics generated within OpenWhisk URL: https://github.com/apache/incubator-openwhisk/pull/3884#issuecomment-405570857 @vvraskin @mhenke1 would you review this? @chetanmeh this is very useful thanks for doing this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah removed a comment on issue #3884: Document metrics generated within OpenWhisk
rabbah removed a comment on issue #3884: Document metrics generated within OpenWhisk URL: https://github.com/apache/incubator-openwhisk/pull/3884#issuecomment-405566993 Fantastic! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3884: Document metrics generated within OpenWhisk
rabbah commented on issue #3884: Document metrics generated within OpenWhisk URL: https://github.com/apache/incubator-openwhisk/pull/3884#issuecomment-405566993 Fantastic! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mdeuser commented on issue #3760: Add API gateway to redo.
mdeuser commented on issue #3760: Add API gateway to redo. URL: https://github.com/apache/incubator-openwhisk/pull/3760#issuecomment-405562162 pg4/1965 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh opened a new pull request #3884: Document metrics generated within OpenWhisk
chetanmeh opened a new pull request #3884: Document metrics generated within OpenWhisk URL: https://github.com/apache/incubator-openwhisk/pull/3884 OpenWhisk generates quite a few metrics via [Kamon](http://kamon.io). This PR aims to document all currently generated metrics such that system administrators can configure the right metrics for monitoring See [here](https://github.com/chetanmeh/incubator-openwhisk/blob/metrics-docs/docs/metrics.md#metric-details) for the rendered HTML version of doc ## Related issue and scope - [ ] I opened an issue to propose and discuss this change (#) ## My changes affect the following components - [ ] API - [ ] Controller - [ ] Message Bus (e.g., Kafka) - [ ] Loadbalancer - [ ] Invoker - [ ] Intrinsic actions (e.g., sequences, conductors) - [ ] Data stores (e.g., CouchDB) - [ ] Tests - [ ] Deployment - [ ] CLI - [ ] General tooling - [x] Documentation ## Types of changes - [ ] Bug fix (generally a non-breaking change which closes an issue). - [ ] Enhancement or new feature (adds new functionality). - [ ] Breaking change (a bug fix or enhancement which changes existing behavior). ## Checklist: - [x] I signed an [Apache CLA](https://github.com/apache/incubator-openwhisk/blob/master/CONTRIBUTING.md). - [x] I reviewed the [style guides](https://github.com/apache/incubator-openwhisk/wiki/Contributing:-Git-guidelines#code-readiness) and followed the recommendations (Travis CI will check :). - [ ] I added tests to cover my changes. - [ ] My changes require further changes to the documentation. - [ ] I updated the documentation where necessary. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jthomas commented on issue #3725: Add Ruby2.5 runtime support
jthomas commented on issue #3725: Add Ruby2.5 runtime support URL: https://github.com/apache/incubator-openwhisk/pull/3725#issuecomment-405538084 https://github.com/apache/incubator-openwhisk-runtime-ruby repo is now available. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3882: Use invoker id for metrics instead of the hostname.
codecov-io edited a comment on issue #3882: Use invoker id for metrics instead of the hostname. URL: https://github.com/apache/incubator-openwhisk/pull/3882#issuecomment-405291454 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=h1) Report > Merging [#3882](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `4.82%`. > The diff coverage is `0%`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/graphs/tree.svg?width=650=pr=l0YmsiSAso=150)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3882 +/- ## == - Coverage 75.78% 70.96% -4.83% == Files 145 145 Lines6897 6901 +4 Branches 418 412 -6 == - Hits 5227 4897 -330 - Misses 1670 2004 +334 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=tree) | Coverage Δ | | |---|---|---| | [...on/scala/src/main/scala/whisk/common/Logging.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvbW1vbi9Mb2dnaW5nLnNjYWxh) | `83.15% <0%> (-3.66%)` | :arrow_down: | | [...er/src/main/scala/whisk/core/invoker/Invoker.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvaW52b2tlci9JbnZva2VyLnNjYWxh) | `0% <0%> (ø)` | :arrow_up: | | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0% <0%> (-95.08%)` | :arrow_down: | | [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0% <0%> (-92.6%)` | :arrow_down: | | [...whisk/core/database/cosmosdb/CosmosDBSupport.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJTdXBwb3J0LnNjYWxh) | `0% <0%> (-81.82%)` | :arrow_down: | | [...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlUHJvdmlkZXIuc2NhbGE=) | `0% <0%> (-58.83%)` | :arrow_down: | | [...la/whisk/core/database/cosmosdb/CosmosDBUtil.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJVdGlsLnNjYWxh) | `92% <0%> (-4%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=footer). Last update [e96c1bb...2175700](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3882: Use invoker id for metrics instead of the hostname.
codecov-io edited a comment on issue #3882: Use invoker id for metrics instead of the hostname. URL: https://github.com/apache/incubator-openwhisk/pull/3882#issuecomment-405291454 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=h1) Report > Merging [#3882](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `5.63%`. > The diff coverage is `0%`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/graphs/tree.svg?src=pr=650=l0YmsiSAso=150)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3882 +/- ## == - Coverage 75.78% 70.14% -5.64% == Files 145 145 Lines6897 6901 +4 Branches 418 412 -6 == - Hits 5227 4841 -386 - Misses 1670 2060 +390 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=tree) | Coverage Δ | | |---|---|---| | [...er/src/main/scala/whisk/core/invoker/Invoker.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvaW52b2tlci9JbnZva2VyLnNjYWxh) | `0% <0%> (ø)` | :arrow_up: | | [...on/scala/src/main/scala/whisk/common/Logging.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvbW1vbi9Mb2dnaW5nLnNjYWxh) | `83.15% <0%> (-3.66%)` | :arrow_down: | | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0% <0%> (-95.08%)` | :arrow_down: | | [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0% <0%> (-92.6%)` | :arrow_down: | | [...whisk/core/database/cosmosdb/CosmosDBSupport.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJTdXBwb3J0LnNjYWxh) | `0% <0%> (-81.82%)` | :arrow_down: | | [...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlUHJvdmlkZXIuc2NhbGE=) | `0% <0%> (-58.83%)` | :arrow_down: | | [...whisk/connector/kafka/KafkaMessagingProvider.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2Nvbm5lY3Rvci9rYWZrYS9LYWZrYU1lc3NhZ2luZ1Byb3ZpZGVyLnNjYWxh) | `23.07% <0%> (-46.16%)` | :arrow_down: | | [...whisk/connector/kafka/KafkaProducerConnector.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2Nvbm5lY3Rvci9rYWZrYS9LYWZrYVByb2R1Y2VyQ29ubmVjdG9yLnNjYWxh) | `22.22% <0%> (-36.12%)` | :arrow_down: | | [...whisk/connector/kafka/KafkaConsumerConnector.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2Nvbm5lY3Rvci9rYWZrYS9LYWZrYUNvbnN1bWVyQ29ubmVjdG9yLnNjYWxh) | `33.33% <0%> (-29.63%)` | :arrow_down: | | ... and [5 more](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=footer). Last update [e96c1bb...2175700](https://codecov.io/gh/apache/incubator-openwhisk/pull/3882?src=pr=lastupdated). Read the [comment
[GitHub] chetanmeh commented on a change in pull request #3882: Use invoker id for metrics instead of the hostname.
chetanmeh commented on a change in pull request #3882: Use invoker id for metrics instead of the hostname. URL: https://github.com/apache/incubator-openwhisk/pull/3882#discussion_r202908636 ## File path: common/scala/src/main/scala/whisk/common/Logging.scala ## @@ -212,6 +214,15 @@ object MetricEmitter { val metrics = Kamon.metrics Review comment: Not sure if it would be safe to add to `MetricEmitter` as first reference to `MetricEmitter` object itself would trigger loading of `metrics` and we would probably need to have config provided and Kamon started before that happens. So would be safer to have a new object hosting this method This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202899807 ## File path: common/scala/src/main/scala/whisk/core/database/CouchDbRestStore.scala ## @@ -513,7 +512,7 @@ class CouchDbRestStore[DocumentAbstraction <: DocumentSerializer](dbProtocol: St .getOrElse(Future.successful(true)) // For CouchDB it is expected that the entire document is deleted. override def shutdown(): Unit = { -Await.ready(client.shutdown(), 1.minute) +client.shutdown() Review comment: Is this change required? `client.shutdown()` returns a `Future` so needs `Await` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202901242 ## File path: common/scala/src/main/scala/whisk/http/PoolingRestClient.scala ## @@ -43,29 +45,36 @@ class PoolingRestClient( host: String, port: Int, queueSize: Int, - httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None)( - implicit system: ActorSystem) { + httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None, + timeout: Option[FiniteDuration] = None)(implicit system: ActorSystem) { require(protocol == "http" || protocol == "https", "Protocol must be one of { http, https }.") protected implicit val context: ExecutionContext = system.dispatcher protected implicit val materializer: ActorMaterializer = ActorMaterializer() + //if specified, override the ClientConnection idle-timeout value + private val timeoutSettings = +ConnectionPoolSettings(system.settings.config) + .withConnectionSettings(if (timeout.isDefined) { +ClientConnectionSettings(system.settings.config) + .withIdleTimeout(timeout.get) + } else { ClientConnectionSettings(system.settings.config) }) Review comment: Alternative ``` scala private val timeoutSettings = { val ps = ConnectionPoolSettings(system.settings.config) timeout.map(t => ps.withUpdatedConnectionSettings(_.withIdleTimeout(t))).getOrElse(ps) } ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202898249 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202897095 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202903907 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() Review comment: `PoolingRestClient.shutdown` returns a `Future` while `ContainerClient.close` is defined to return a Unit. Should we wait for the result completion or change contract for `ContainerClient`? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202898726 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202896946 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202898803 ## File path: common/scala/src/main/scala/whisk/core/containerpool/ContainerClient.scala ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool + +import akka.actor.ActorSystem +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.http.scaladsl.marshalling.Marshal +import akka.http.scaladsl.model.HttpMethods +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.MessageEntity +import akka.http.scaladsl.model.StatusCodes +import akka.http.scaladsl.model.headers.Connection +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.stream.StreamTcpException +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import akka.util.ByteString +import scala.concurrent.Await +import scala.concurrent.ExecutionContext +import scala.concurrent.Future +import scala.concurrent.TimeoutException +import scala.concurrent.duration._ +import scala.util.Try +import spray.json._ +import whisk.common.Logging +import whisk.common.TransactionId +import whisk.core.entity.ActivationResponse.ContainerHttpError +import whisk.core.entity.ActivationResponse._ +import whisk.core.entity.ByteSize +import whisk.core.entity.size.SizeLong +import whisk.http.PoolingRestClient + +trait ContainerClient { + def close(): Unit + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] + +} + +/** + * This HTTP client is used only in the invoker to communicate with the action container. + * It allows to POST a JSON object and receive JSON object back; that is the + * content type and the accept headers are both 'application/json. + * The reason we still use this class for the action container is a mysterious hang + * in the Akka http client where a future fails to properly timeout and we have not + * determined why that is. + * + * @param hostname the host name + * @param timeout the timeout in msecs to wait for a response + * @param maxResponse the maximum size in bytes the connection will accept + * @param queueSize once all connections are used, how big of queue to allow for additional requests + * @param retryInterval duration between retries for TCP connection errors + */ +protected class PoolingContainerClient( + hostname: String, + port: Int, + timeout: FiniteDuration, + maxResponse: ByteSize, + queueSize: Int, + retryInterval: FiniteDuration = 100.milliseconds)(implicit logging: Logging, as: ActorSystem) +extends PoolingRestClient("http", hostname, port, queueSize, timeout = Some(timeout)) +with ContainerClient +with AutoCloseable { + + def close() = shutdown() + + /** + * Posts to hostname/endpoint the given JSON object. + * Waits up to timeout before aborting on a good connection. + * If the endpoint is not ready, retry up to timeout. + * Every retry reduces the available timeout so that this method should not + * wait longer than the total timeout (within a small slack allowance). + * + * @param endpoint the path the api call relative to hostname + * @param body the JSON value to post (this is usually a JSON objecT) + * @param retry whether or not to retry on connection failure + * @return Left(Error Message) or Right(Status Code, Response as UTF-8 String) + */ + def post(endpoint: String, body: JsValue, retry: Boolean)( +implicit tid: TransactionId): Future[Either[ContainerHttpError, ContainerResponse]] = { + +//create the request +val req = Marshal(body).to[MessageEntity].map { b => + //DO NOT reuse the connection (in case of paused containers) + //For details on Connection: Close handling, see: + // - https://doc.akka.io/docs/akka-http/current/common/http-model.html#http-headers + // -
[GitHub] chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils
chetanmeh commented on a change in pull request #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#discussion_r202901499 ## File path: common/scala/src/main/scala/whisk/http/PoolingRestClient.scala ## @@ -43,29 +45,36 @@ class PoolingRestClient( host: String, port: Int, queueSize: Int, - httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None)( - implicit system: ActorSystem) { + httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None, + timeout: Option[FiniteDuration] = None)(implicit system: ActorSystem) { require(protocol == "http" || protocol == "https", "Protocol must be one of { http, https }.") protected implicit val context: ExecutionContext = system.dispatcher protected implicit val materializer: ActorMaterializer = ActorMaterializer() + //if specified, override the ClientConnection idle-timeout value + private val timeoutSettings = +ConnectionPoolSettings(system.settings.config) + .withConnectionSettings(if (timeout.isDefined) { +ClientConnectionSettings(system.settings.config) + .withIdleTimeout(timeout.get) + } else { ClientConnectionSettings(system.settings.config) }) + // Creates or retrieves a connection pool for the host. private val pool = if (protocol == "http") { -Http().cachedHostConnectionPool[Promise[HttpResponse]](host = host, port = port) +Http().cachedHostConnectionPool[Promise[HttpResponse]](host = host, port = port, settings = timeoutSettings) } else { -Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]](host = host, port = port) +Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]](host = host, port = port, settings = timeoutSettings) } - // Additional queue in case all connections are busy. Should hardly ever be - // filled in practice but can be useful, e.g., in tests starting many - // asynchronous requests in a very short period of time. Review comment: Docs can be retained This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3812: ContainerClient + akka http alternative to HttpUtils
codecov-io edited a comment on issue #3812: ContainerClient + akka http alternative to HttpUtils URL: https://github.com/apache/incubator-openwhisk/pull/3812#issuecomment-401212372 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=h1) Report > Merging [#3812](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-openwhisk/commit/e96c1bbd5a0f54c923f0153116f7c1f165275e94?src=pr=desc) will **decrease** coverage by `4.6%`. > The diff coverage is `55.55%`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/graphs/tree.svg?token=l0YmsiSAso=pr=150=650)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3812 +/- ## == - Coverage 75.78% 71.17% -4.61% == Files 145 146 +1 Lines6897 6939 +42 Branches 418 432 +14 == - Hits 5227 4939 -288 - Misses 1670 2000 +330 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=tree) | Coverage Δ | | |---|---|---| | [...la/whisk/core/containerpool/ContainerFactory.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJGYWN0b3J5LnNjYWxh) | `100% <ø> (ø)` | :arrow_up: | | [...la/src/main/scala/whisk/core/mesos/MesosTask.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvbWVzb3MvTWVzb3NUYXNrLnNjYWxh) | `86.11% <ø> (ø)` | :arrow_up: | | [...containerpool/kubernetes/KubernetesContainer.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9rdWJlcm5ldGVzL0t1YmVybmV0ZXNDb250YWluZXIuc2NhbGE=) | `91.66% <ø> (ø)` | :arrow_up: | | [...sk/core/containerpool/docker/DockerContainer.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9kb2NrZXIvRG9ja2VyQ29udGFpbmVyLnNjYWxh) | `79.16% <0%> (+2.13%)` | :arrow_up: | | [...n/scala/whisk/core/database/CouchDbRestStore.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvQ291Y2hEYlJlc3RTdG9yZS5zY2FsYQ==) | `73.23% <100%> (ø)` | :arrow_up: | | [.../src/main/scala/whisk/http/PoolingRestClient.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2h0dHAvUG9vbGluZ1Jlc3RDbGllbnQuc2NhbGE=) | `90.9% <100%> (+1.62%)` | :arrow_up: | | [...ain/scala/whisk/core/containerpool/HttpUtils.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9IdHRwVXRpbHMuc2NhbGE=) | `69.81% <14.28%> (+18.9%)` | :arrow_up: | | [...ain/scala/whisk/core/containerpool/Container.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXIuc2NhbGE=) | `76.56% <25%> (+0.37%)` | :arrow_up: | | [...ala/whisk/core/containerpool/ContainerClient.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJDbGllbnQuc2NhbGE=) | `67.5% <67.5%> (ø)` | | | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL3doaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0% <0%> (-100%)` | :arrow_down: | | ... and [7 more](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=footer). Last update [e96c1bb...fb0b337](https://codecov.io/gh/apache/incubator-openwhisk/pull/3812?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).