Last week I had a Slack call with Rodric around AttachmentStore PR and following possible enhancements around attachment handling in OpenWhisk were discussed. These enhancements would enable efficient handling of code content specially for large actions
A - Attachment Inlining ------------------------------ Currently code related to action can be stored in following form 1. Plaintext string - Code is stored as is in plain text form in `code` attribute 2. Base64 encoded zip content - Code is stored as base64 encoded zipped content in `code` attribute with `binary` set to true 3. CouchDB attachment - Code content is stored as raw bytes via CouchDB attachment support [1] In all cases when Invoker invokes the action the code content is passed as string as part of initializer json. Also when action is created the code is passed inline as part of json. Going forward (specially useful with switch to external AttachmentStore) the code content would always be stored as attachment (PR #2847). For such cases it would better to have a way where smaller code can be stored inlined (as being done for non java cases currently) to avoid extra calls to AttachmentStore. This inlining logic would be an internal detail of AttachmentStore and would be based on configurable maxInlineSize configuration irrespective of action kind. B - Streaming Action Code - Action Creation ---------------------------------------------------------- Currently the action is provided inline string attribute in json payload POSTed to namespaces/<package>/actions/<actionName> endpoint. Due to this whole code content is processed in memory as byte array. Compared to this the ArtifactStore abstraction supports streams for creating and reading attachments. For supporting large action code it would better if we enable streaming of action code from client to ArtifactStore without storing it as byte array within heap. This can be done by supporting multi-part upload [2] for action endpoint and then just passing the byteSource to ArtifactStore C - Streaming Action Code - Action Execution ------------------------------------------------------------- This compliments the #B option. Currently action code is passed as part of json in action initializer call. Going forward with support of AttachmenetStore we can make use of constructs like signed url [3] (suppored in most stores like S3, IBM COS and Azure Blob Storage). Here instead of passing the action code in json the Invoker would obtained a signed url which is then passed to container and internally container directly streams the url content. Kindly share your thoughts/comment around this topic Chetan Mehrotra [1] http://docs.couchdb.org/en/2.0.0/api/document/attachments.html [2] https://doc.akka.io/docs/akka-http/current/routing-dsl/directives/file-upload-directives/fileUpload.html [3] https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html
