AlexanderKaraberov opened a new pull request #1195: Add support for bulk get 
with Accept Content Type: "multipart/mixed", "multipart/related"
URL: https://github.com/apache/couchdb/pull/1195
 
 
   ## Overview
   
   CouchDB currently has support for `_bulk_get` requests which are intended to 
improve performance of client pull replications, by allowing the client to 
request multiple documents in one request. But current implementation has some 
limitations namely: attachment bodies can only be encoded inline (base64) not 
as MIME multipart bodies. Another issue which was much more important in our 
case: current _bulk_get implementation in CouchDB is not compatible with the 
[Couchbase Lite 
1.4.x](https://github.com/couchbase/couchbase-lite-ios/tree/release/1.4.1) for 
iOS and Android because their `BulkDownloader` accepts [multipart/related 
content 
type](https://github.com/couchbase/couchbase-lite-ios/blob/release/1.4.1/Source/CBLBulkDownloader.m#L85)
 and not `application/json`. 
   This PR adds support for `multipart/related`/`multipart/mixed` _bulk_get 
response which is compatible with [Couchbase Lite/Sync 
Gateway](https://github.com/couchbase/sync_gateway/wiki/Bulk-GET). Code is 
pretty much based on the [rcouch](https://github.com/rcouch) implementation 
with some changes to make it compatible with current CouchDB 2 architecture and 
also doesn't use any external libraries. 
   Logic is pretty simple: depending on the `case 
MochiReq:accepts_content_type("multipart/mixed")` check I pick either a default 
CouchDB's _bulk_get implementation or a multipart one. I specifically added a 
`send_doc_multipart` function so that not to break an existing 
`send_docs_multipart` one which wasn't compatible with the logic.
   We already tested this with some of the big production CouchDB databases and 
speedup was very noticeable. I think this feature will be very useful because 
there are a lot of Couchbase Lite users who can't benefit from the _bulk_get 
because of `application/json` only response. Plus this is a completely additive 
change which doesn't break/change existing functionality. Test suite is based 
on the `chttpd_db_bulk_get_test`. If some of the tests are missing or there are 
some issues with the implementation - feel free to point and I will fix them.
   
   ## Testing recommendations
   For testing create a `db1`, put some docs there: 
   ```
   curl -u admin:rm -X PUT http://127.0.0.1:5983/db1
   curl -u admin:rm -H "Content-Type: application/json" -X PUT 
http://127.0.0.1:5983/db1/doc1 -d '{"type":"a"}'
   curl -u admin:rm -H "Content-Type: application/json" -X PUT 
http://127.0.0.1:5983/db1/doc2 -d '{"type":"a"}'
   curl -u admin:rm -H "Content-Type: application/json" -X PUT 
http://127.0.0.1:5983/db1/doc3 -d '{"type":"b"}'
   ```
   Also add attachments for some of the docs (.png images or whatnot). After 
this you can execute similar curl command:
   ```
   curl -u "admin:rm" -X POST -H "Content-Type: application/json" -H "Accept: 
multipart/related" -H "X-Accept-Part-Encoding: gzip"  -o "response.json" -d 
'{"docs": [{"id":"doc1", "rev": "1-8d0539dc5c5fe98d4069d2d0b06d9fa6"},{"id": 
"doc2", "rev":"1-b59ac7ccf4c21b26f6a4e1d72c509f23"},{"id": "doc3", "rev": 
"1-fc9b2660a790d8d98589102162ccb63d"}]}' 
http://127.0.0.1:5983/db1/_bulk_get?revs=true&attachments=true
   ```
   Then in the response.json you can observe something similar to this:
   ```json
   -----------------------------abe834719cdf618f78719d2a8f3edd06
   Content-Type: application/json
   
   {"_id":"doc1","_rev":"1-8d0539dc5c5fe98d4069d2d0b06d9fa6","fp_owner":"global"
   -----------------------------abe834719cdf618f78719d2a8f3edd06
   X-Doc-Id: _design/briefcase
   X-Rev-Id: 3-9df87eb32d30ce1e01fe4ca121ac725e
   Content-Type: multipart/related; 
boundary=---------------------------edc31bb71a98be453c3ea40d4f96019e
   
   -----------------------------edc31bb71a98be453c3ea40d4f96019e
   Content-Type: application/json
   
   
{"_id":"doc2","_rev":"3-9df87eb32d30ce1e01fe4ca121ac725e","fp_owner":"global"}
   -----------------------------edc31bb71a98be453c3ea40d4f96019e
   Content-Disposition: attachment; filename="IMG_0361.PNG"
   Content-Type: image/png
   Content-Length: 2466384
   
   ?PNG
   
   
   IHDRe   ????aiCCPkCGColorSpaceDisplayP3(?c``RI,(?aa``??+)
   rwR???R`????b
   ????>@%0|????/???:%5?I?^??b????D?0?????d ??S??JS?l?????):
   
?b?C?@?$?XMH?3?}?VH?H?????IBOGbC?n????J?c?%??V??h???????G`(?*x?%??(?30????s
 8,?? ??30??????n???~??@?\;b??
'v$%?????)-????r?H?@=??i?F`yF'?{??Vc``????w???????w1P??y!e?5????iTXtXML:com.adobe.xmp<x:xmpmeta
 xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
      <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";>
         <rdf:Description rdf:about=""
               xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/";
               xmlns:exif="http://ns.adobe.com/exif/1.0/";>
            <photoshop:DateCreated>2018-01-25T11:00:11</photoshop:DateCreated>
            <exif:UserComment>Screenshot</exif:UserComment>
         </rdf:Description>
      </rdf:RDF>
   </x:xmpmeta>
   ?8?@IDATx????ga'~????~?o????#I+??I?Y??|?8??d0?9? &???c R
   
g8rp??C?Q"??H#Y?u??GR???R???=?n?^.uMu?????OWmy?z???`?????8?p??8?p??8?p??8?????
   -----------------------------edc31bb71a98be453c3ea40d4f96019e--
   
   -----------------------------abe834719cdf618f78719d2a8f3edd06--
   ```
   This format is completely compatible with Couchbase Lite and also Sync 
Gateway implementation.
   
   ## Checklist
   
   - [ ] Code is written and works correctly;
   - [ ] Changes are covered by tests;
   - [ ] Documentation reflects the changes;
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to