AlexanderKaraberov opened a new pull request #1195: Add support for bulk get with Accept Content Type: "multipart/mixed", "multipart/related" URL: https://github.com/apache/couchdb/pull/1195 ## Overview CouchDB currently has support for `_bulk_get` requests which are intended to improve performance of client pull replications, by allowing the client to request multiple documents in one request. But current implementation has some limitations namely: attachment bodies can only be encoded inline (base64) not as MIME multipart bodies. Another issue which was much more important in our case: current _bulk_get implementation in CouchDB is not compatible with the [Couchbase Lite 1.4.x](https://github.com/couchbase/couchbase-lite-ios/tree/release/1.4.1) for iOS and Android because their `BulkDownloader` accepts [multipart/related content type](https://github.com/couchbase/couchbase-lite-ios/blob/release/1.4.1/Source/CBLBulkDownloader.m#L85) and not `application/json`. This PR adds support for `multipart/related`/`multipart/mixed` _bulk_get response which is compatible with [Couchbase Lite/Sync Gateway](https://github.com/couchbase/sync_gateway/wiki/Bulk-GET). Code is pretty much based on the [rcouch](https://github.com/rcouch) implementation with some changes to make it compatible with current CouchDB 2 architecture and also doesn't use any external libraries. Logic is pretty simple: depending on the `case MochiReq:accepts_content_type("multipart/mixed")` check I pick either a default CouchDB's _bulk_get implementation or a multipart one. I specifically added a `send_doc_multipart` function so that not to break an existing `send_docs_multipart` one which wasn't compatible with the logic. We already tested this with some of the big production CouchDB databases and speedup was very noticeable. I think this feature will be very useful because there are a lot of Couchbase Lite users who can't benefit from the _bulk_get because of `application/json` only response. Plus this is a completely additive change which doesn't break/change existing functionality. Test suite is based on the `chttpd_db_bulk_get_test`. If some of the tests are missing or there are some issues with the implementation - feel free to point and I will fix them. ## Testing recommendations For testing create a `db1`, put some docs there: ``` curl -u admin:rm -X PUT http://127.0.0.1:5983/db1 curl -u admin:rm -H "Content-Type: application/json" -X PUT http://127.0.0.1:5983/db1/doc1 -d '{"type":"a"}' curl -u admin:rm -H "Content-Type: application/json" -X PUT http://127.0.0.1:5983/db1/doc2 -d '{"type":"a"}' curl -u admin:rm -H "Content-Type: application/json" -X PUT http://127.0.0.1:5983/db1/doc3 -d '{"type":"b"}' ``` Also add attachments for some of the docs (.png images or whatnot). After this you can execute similar curl command: ``` curl -u "admin:rm" -X POST -H "Content-Type: application/json" -H "Accept: multipart/related" -H "X-Accept-Part-Encoding: gzip" -o "response.json" -d '{"docs": [{"id":"doc1", "rev": "1-8d0539dc5c5fe98d4069d2d0b06d9fa6"},{"id": "doc2", "rev":"1-b59ac7ccf4c21b26f6a4e1d72c509f23"},{"id": "doc3", "rev": "1-fc9b2660a790d8d98589102162ccb63d"}]}' http://127.0.0.1:5983/db1/_bulk_get?revs=true&attachments=true ``` Then in the response.json you can observe something similar to this: ```json -----------------------------abe834719cdf618f78719d2a8f3edd06 Content-Type: application/json {"_id":"doc1","_rev":"1-8d0539dc5c5fe98d4069d2d0b06d9fa6","fp_owner":"global" -----------------------------abe834719cdf618f78719d2a8f3edd06 X-Doc-Id: _design/briefcase X-Rev-Id: 3-9df87eb32d30ce1e01fe4ca121ac725e Content-Type: multipart/related; boundary=---------------------------edc31bb71a98be453c3ea40d4f96019e -----------------------------edc31bb71a98be453c3ea40d4f96019e Content-Type: application/json {"_id":"doc2","_rev":"3-9df87eb32d30ce1e01fe4ca121ac725e","fp_owner":"global"} -----------------------------edc31bb71a98be453c3ea40d4f96019e Content-Disposition: attachment; filename="IMG_0361.PNG" Content-Type: image/png Content-Length: 2466384 ?PNG IHDR e ? ??? aiCCPkCGColorSpaceDisplayP3 (?c``RI,(?aa``??+) rwR???R`???? b ????>@% 0|????/???:%5?I?^??b????D?0? ????d ??S? ?JS?l?????): ?b?C?@?$?XMH?3?}?VH?H?????IBOGbC?n????J? c?%??V??h???????G`(?*x?%??(?30?? ??s 8,?? ??30??????n???~??@?\;b ?? 'v$%?????)-????r?H? @=??i?F`yF '?{??Vc``????w???????w1P? ?y !e?5??? ?iTXtXML:com.adobe.xmp <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0"> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";> <rdf:Description rdf:about="" xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"; xmlns:exif="http://ns.adobe.com/exif/1.0/";> <photoshop:DateCreated>2018-01-25T11:00:11</photoshop:DateCreated> <exif:UserComment>Screenshot</exif:UserComment> </rdf:Description> </rdf:RDF> </x:xmpmeta> ?8? @ IDATx???? ga'~?? ? ?~?o????#I+??I?Y??|?8??d0?9? &???c R g8rp? ?C?Q"??H#Y?u??GR???R ???=?n?^.uMu ?????OWmy?z???`? ??? ? 8?p? ? 8?p? ? 8?p? ? 8????? -----------------------------edc31bb71a98be453c3ea40d4f96019e-- -----------------------------abe834719cdf618f78719d2a8f3edd06-- ``` This format is completely compatible with Couchbase Lite and also Sync Gateway implementation. ## Checklist - [ ] Code is written and works correctly; - [ ] Changes are covered by tests; - [ ] Documentation reflects the changes;
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
