jiangpengcheng opened a new pull request #4963: URL: https://github.com/apache/openwhisk/pull/4963
This PR provides a MongoDBArtifactStore implementation to enable using MongoDB for storing subjects, whisks and activation in [MongoDB](https://www.mongodb.com/)， and mostly comes from old PR: https://github.com/apache/openwhisk/pull/3570. Since lots of codes are inspired by https://github.com/chetanmeh/openwhisk-mongo, so I added @chetanmeh as co-author ## Description Some users may want to use other database backends instead of CouchDB to store entities, this PR provides MongoDB as an alternative choice for these users ## MongoDB driver MongoDB provides [official drivers](https://docs.mongodb.com/ecosystem/drivers/) for many languages, in this case, we pick the [mongo-scala-driver](http://mongodb.github.io/mongo-scala-driver/2.2/), this driver provides an idiomatic Scala API that is built on top of the MongoDB Async Java driver. ## Design Considerations ### Data Model The data scheme in MongoDB is just like CouchDB except that MongoDB doesn't have a `_rev` field, below is the data of `whisk.system/invokerHealthTestAction0` in CouchDB and MongoDB: <table> <tr> <td>CouchDB</td> <td>MongoDB</td> </tr> <tr> <td> <pre> { "_id": "whisk.system/invokerHealthTestAction0", "_rev": "68-e72440f911c64ab11441c09e730e5ab8", "name": "invokerHealthTestAction0", "publish": false, "annotations": [], "version": "0.0.1", "updated": 1524476933182, "entityType": "action", "exec": { "kind": "nodejs:6", "code": "function main(params) { return params; }", "binary": false }, "parameters": [], "limits": { "timeout": 60000, "memory": 256, "logs": 10 }, "namespace": "whisk.system" } </pre> </td> <td> <pre> { "_id" : "whisk.system/invokerHealthTestAction0", "name" : "invokerHealthTestAction0", "publish" : false, "annotations" : "[ ]", "version" : "0.0.1", "updated" : NumberLong("1524473794826"), "entityType" : "action", "exec" : { "kind" : "nodejs:6", "code" : "function main(params) { return params; }", "binary" : false }, "parameters" : "[ ]", "limits" : { "timeout" : 60000, "memory" : 256, "logs" : 10 }, "namespace" : "whisk.system" } </pre> </td> </tr> </table> - *Since the `annotations` and `parameters` fields may using arbitrary strcut and MongoDB doesn't support `$` as the first char for field name, so it will convert these two fields to string, and convert it back when get them via openwhisk, this is transparent to users* - *`_rev` field will not be generated automatically in MongoDB, so it is calculted and inserted in code explicitly* Regarding the `_rev` field, there is no alternative way in MongoDB to do the same thing, so the `MongoDbStore` calculate a value according to the document's content manually, and make it as the `_rev` field. ### Attachment MongoDB use [GridFS](https://docs.mongodb.com/manual/core/gridfs/index.html) to store and retrieve files that exceed the BSON-document size limit of 16 MB. Attachment in MongoDB is stored in a separate collection with a independent `_id`, this PR use the `doc._id + doc.file_name` as the attachment's `_id` field, then we can find the relative attachment easily. ## Progress ### Finished Work - [x] Basic Usage(create、update、query、delete) - [x] Attachment Support - [x] Automated deployment of MongoDB using Ansible - [x] CI Integration - [x] Documents ### FurtherWork - [ ] Add MongoDB support for wskadmin - [ ] Support batch insert ## My changes affect the following components - [x] Data stores (e.g., CouchDB) - [x] Tests - [x] Deployment - [ ] CLI - [x] Documentation ## Types of changes - [x] Enhancement or new feature (adds new functionality). ## Checklist: - [x] I signed an [Apache CLA](https://github.com/apache/incubator-openwhisk/blob/master/CONTRIBUTING.md). - [x] I reviewed the [style guides](https://github.com/apache/incubator-openwhisk/wiki/Contributing:-Git-guidelines#code-readiness) and followed the recommendations (Travis CI will check :). - [x] I added tests to cover my changes. - [ ] My changes require further changes to the documentation. - [x] I updated the documentation where necessary. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
