[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955881#comment-15955881
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 7e48bda4459cc8e4dbb8bd86966792f533571d83 in couchdb-couch's branch 
refs/heads/COUCHDB-3288-remove-public-db-record from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=7e48bda ]

Allow limiting maximum document body size

Configuration is via the `couchdb.max_document_size`. In the past that
was implemented as a maximum http request body size and this finally implements
it by actually checking a document's body size.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955851#comment-15955851
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 3b15107df83a16a26dbc6c06a1a080437cb558b8 in couchdb-fabric's branch 
refs/heads/COUCHDB-3287-pluggable-storage-engines from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-fabric.git;h=3b15107 ]

Allow limiting maximum document body size

Update doc function to check and validate document body sizes

Main implementation is in PR:

 https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955843#comment-15955843
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 3b15107df83a16a26dbc6c06a1a080437cb558b8 in couchdb-fabric's branch 
refs/heads/COUCHDB-3288-remove-public-db-record from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-fabric.git;h=3b15107 ]

Allow limiting maximum document body size

Update doc function to check and validate document body sizes

Main implementation is in PR:

 https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955834#comment-15955834
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 64958096d4f9a940c01cbc472da5265f349c9545 in couchdb-couch-replicator's 
branch refs/heads/COUCHDB-3287-pluggable-storage-engines from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=6495809
 ]

Fix unit test after renaming max_document_size config parameter

`couchdb.max_document_size` was renamed to `httpd.max_http_request_size`

The unit tests was testing how replicator behaves when faced with reduced
request size configuration on the target.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955833#comment-15955833
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 30915e3309fb30c2164e668d33dbd393e77925c0 in couchdb-couch-replicator's 
branch refs/heads/COUCHDB-3287-pluggable-storage-engines from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=30915e3
 ]

Remove unused mp_parse_doc function from replicator

It was left accidentally when merging Cloudant's dbcore work.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955830#comment-15955830
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 64958096d4f9a940c01cbc472da5265f349c9545 in couchdb-couch-replicator's 
branch refs/heads/COUCHDB-3288-remove-public-db-record from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=6495809
 ]

Fix unit test after renaming max_document_size config parameter

`couchdb.max_document_size` was renamed to `httpd.max_http_request_size`

The unit tests was testing how replicator behaves when faced with reduced
request size configuration on the target.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955829#comment-15955829
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 30915e3309fb30c2164e668d33dbd393e77925c0 in couchdb-couch-replicator's 
branch refs/heads/COUCHDB-3288-remove-public-db-record from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=30915e3
 ]

Remove unused mp_parse_doc function from replicator

It was left accidentally when merging Cloudant's dbcore work.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955823#comment-15955823
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 398c30e8785c3cd880d7d9788d25810dfe626c18 in couchdb-couch-mrview's 
branch refs/heads/COUCHDB-3287-pluggable-storage-engines from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=398c30e ]

Allow limiting maximum document body size

This is a companion commit to this one:

https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955808#comment-15955808
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 398c30e8785c3cd880d7d9788d25810dfe626c18 in couchdb-couch-mrview's 
branch refs/heads/COUCHDB-3288-remove-public-db-record from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=398c30e ]

Allow limiting maximum document body size

This is a companion commit to this one:

https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955803#comment-15955803
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit a1470e3bdbcb4b98d9cc7f5dc3641a2b008df16b in couchdb-chttpd's branch 
refs/heads/COUCHDB-3287-pluggable-storage-engines from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=a1470e3 ]

Rename max_document_size to max_http_request_size

`max_document_size` is implemented as `max_http_request_size`. There was no
real check for document size. In some cases the implementation was close enough
of a proxy (PUT-ing and GET-ing single docs), but in some edge cases, like
_bulk_docs requests the discrepancy between request size and document size
could be rather large.

The section was changed accordingly from `couchdb` to `httpd`. `httpd` was
chosen as it applies to both clustered as well as local interface.

There is a parallel effort to implement an actual max_document_size check. The
set of commit should be merged close enough together to allow for a backwards
compatible transition.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955800#comment-15955800
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit a1470e3bdbcb4b98d9cc7f5dc3641a2b008df16b in couchdb-chttpd's branch 
refs/heads/COUCHDB-3288-remove-public-db-record from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=a1470e3 ]

Rename max_document_size to max_http_request_size

`max_document_size` is implemented as `max_http_request_size`. There was no
real check for document size. In some cases the implementation was close enough
of a proxy (PUT-ing and GET-ing single docs), but in some edge cases, like
_bulk_docs requests the discrepancy between request size and document size
could be rather large.

The section was changed accordingly from `couchdb` to `httpd`. `httpd` was
chosen as it applies to both clustered as well as local interface.

There is a parallel effort to implement an actual max_document_size check. The
set of commit should be merged close enough together to allow for a backwards
compatible transition.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931063#comment-15931063
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user nickva closed the pull request at:

https://github.com/apache/couchdb-documentation/pull/113


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931064#comment-15931064
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit d57f776b4417210cca581bc24bf5ae97dc2de2aa in couchdb-documentation's 
branch refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-documentation.git;h=d57f776 
]

Bring back the correct implementation of max_document_size

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927455#comment-15927455
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-documentation/pull/113

Bring back the correct implementation of max_document_size

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-documentation couchdb-2992

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-documentation/pull/113.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #113


commit 55cad9af960e4d299bbf387a9e57cd8c7e6273c8
Author: Nick Vatamaniuc 
Date:   2017-03-16T04:07:02Z

Bring back the correct implementation of max_document_size

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927400#comment-15927400
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit d9ab78e109a9c4b98e33eceef2f6bb1886d749d7 in couchdb's branch 
refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=d9ab78e ]

Bump couch, fabric, chttpd, couch_mrview to implement max_document_size limit

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927382#comment-15927382
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 398c30e8785c3cd880d7d9788d25810dfe626c18 in couchdb-couch-mrview's 
branch refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=398c30e ]

Allow limiting maximum document body size

This is a companion commit to this one:

https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927383#comment-15927383
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-couch-mrview/pull/69


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927365#comment-15927365
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 3b15107df83a16a26dbc6c06a1a080437cb558b8 in couchdb-fabric's branch 
refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-fabric.git;h=3b15107 ]

Allow limiting maximum document body size

Update doc function to check and validate document body sizes

Main implementation is in PR:

 https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927366#comment-15927366
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-fabric/pull/91


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927360#comment-15927360
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit d1848e6f2288ea9b3758c22f10f75706a87be3b5 in couchdb-chttpd's branch 
refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=d1848e6 ]

Allow limiting maximum document body size

This is the HTTP layer and some tests. The actual checking is done in couch
application's from_json_obj/1 function.

If a document is too large it will return a 413 response code. The error reason
will be the document ID. The intent is to help users identify the document if
they used _bulk_docs endpoint. It will also help replicator skip over documents
which are too large.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927363#comment-15927363
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-chttpd/pull/157


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927356#comment-15927356
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 7e48bda4459cc8e4dbb8bd86966792f533571d83 in couchdb-couch's branch 
refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=7e48bda ]

Allow limiting maximum document body size

Configuration is via the `couchdb.max_document_size`. In the past that
was implemented as a maximum http request body size and this finally implements
it by actually checking a document's body size.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927357#comment-15927357
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-couch/pull/235


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927073#comment-15927073
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-couch-mrview/pull/69

Allow limiting maximum document body size

This is a companion commit to this one:

https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-couch-mrview couchdb-2992

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-couch-mrview/pull/69.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #69


commit 398c30e8785c3cd880d7d9788d25810dfe626c18
Author: Nick Vatamaniuc 
Date:   2017-03-15T21:55:22Z

Allow limiting maximum document body size

This is a companion commit to this one:

https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925154#comment-15925154
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user tonysun83 closed the pull request at:

https://github.com/apache/couchdb-chttpd/pull/114


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924968#comment-15924968
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 30915e3309fb30c2164e668d33dbd393e77925c0 in couchdb-couch-replicator's 
branch refs/heads/63012-scheduler from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=30915e3
 ]

Remove unused mp_parse_doc function from replicator

It was left accidentally when merging Cloudant's dbcore work.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924969#comment-15924969
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 64958096d4f9a940c01cbc472da5265f349c9545 in couchdb-couch-replicator's 
branch refs/heads/63012-scheduler from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=6495809
 ]

Fix unit test after renaming max_document_size config parameter

`couchdb.max_document_size` was renamed to `httpd.max_http_request_size`

The unit tests was testing how replicator behaves when faced with reduced
request size configuration on the target.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923697#comment-15923697
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-fabric/pull/91

Allow limiting maximum document body size

Update doc function to check and validate document body sizes

Main implementation is in PR:

 https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-fabric couchdb-2992

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-fabric/pull/91.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #91


commit 3b15107df83a16a26dbc6c06a1a080437cb558b8
Author: Nick Vatamaniuc 
Date:   2017-03-14T06:59:31Z

Allow limiting maximum document body size

Update doc function to check and validate document body sizes

Main implementation is in PR:

 https://github.com/apache/couchdb-couch/pull/235

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907580#comment-15907580
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-chttpd/pull/157

Allow limiting maximum document body size

This is the HTTP layer and some tests. The actual checking is done in couch
application's from_json_obj/1 function.

If a document is too large it will return a 413 response code. The error 
reason
will be the document ID. The intent is to help users identify the document 
if
they used _bulk_docs endpoint. It will also help replicator skip over 
documents
which are too large.

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-chttpd couchdb-2992

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-chttpd/pull/157.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #157


commit 95ecd629f77444dd5fa2820fbc18cccaba350f6d
Author: Nick Vatamaniuc 
Date:   2017-03-13T06:22:19Z

Allow limiting maximum document body size

This is the HTTP layer and some tests. The actual checking is done in couch
application's from_json_obj/1 function.

If a document is too large it will return a 413 response code. The error 
reason
will be the document ID. The intent is to help users identify the document 
if
they used _bulk_docs endpoint. It will also help replicator skip over 
documents
which are too large.

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907575#comment-15907575
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-couch/pull/235

Allow limiting maximum document body size

Configuration is via the `couchdb.max_document_size`. In the past that
was implemented as a maximum http request body size and this finally 
implements
it by actually checking a document's body size.

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-couch couchdb-2992

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-couch/pull/235.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #235


commit c51c3f2bdc8e2f2a135c8363096762607ed33f2c
Author: Nick Vatamaniuc 
Date:   2017-03-13T06:33:57Z

Allow limiting maximum document body size

Configuration is via the `couchdb.max_document_size`. In the past that
was implemented as a maximum http request body size and this finally 
implements
it by actually checking a document's body size.

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901475#comment-15901475
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-documentation/pull/105


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901474#comment-15901474
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 83bd3f83b9c08bfb3b8f27fe0653ba27c056c776 in couchdb-documentation's 
branch refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-documentation.git;h=83bd3f8 
]

Add documentation for the new `max_http_request_size` parameter

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900528#comment-15900528
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-couch/pull/233


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900436#comment-15900436
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 64958096d4f9a940c01cbc472da5265f349c9545 in couchdb-couch-replicator's 
branch refs/heads/master from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=6495809
 ]

Fix unit test after renaming max_document_size config parameter

`couchdb.max_document_size` was renamed to `httpd.max_http_request_size`

The unit tests was testing how replicator behaves when faced with reduced
request size configuration on the target.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900438#comment-15900438
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-couch-replicator/pull/61


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900433#comment-15900433
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit a1470e3bdbcb4b98d9cc7f5dc3641a2b008df16b in couchdb-chttpd's branch 
refs/heads/master from [~vatamane]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=a1470e3 ]

Rename max_document_size to max_http_request_size

`max_document_size` is implemented as `max_http_request_size`. There was no
real check for document size. In some cases the implementation was close enough
of a proxy (PUT-ing and GET-ing single docs), but in some edge cases, like
_bulk_docs requests the discrepancy between request size and document size
could be rather large.

The section was changed accordingly from `couchdb` to `httpd`. `httpd` was
chosen as it applies to both clustered as well as local interface.

There is a parallel effort to implement an actual max_document_size check. The
set of commit should be merged close enough together to allow for a backwards
compatible transition.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900296#comment-15900296
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-couch-replicator/pull/61

Fix unit test after renaming max_document_size config parameter

`couchdb.max_document_size` was renamed to `httpd.max_http_request_size`

The unit tests was testing how replicator behaves when faced with reduced
request size configuration on the target.

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-couch-replicator 
64229-add-new-request-parameter

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-couch-replicator/pull/61.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #61


commit d43aa56c8814dbfd94c8856dc017bfe45047
Author: Nick Vatamaniuc 
Date:   2017-03-07T22:24:00Z

Fix unit test after renaming max_document_size config parameter

`couchdb.max_document_size` was renamed to `httpd.max_http_request_size`

The unit tests was testing how replicator behaves when faced with reduced
request size configuration on the target.

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900268#comment-15900268
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-couch/pull/233

Rename max_document_size to max_http_request_size

`max_document_size` is implemented as `max_http_request_size`. There was no
real check for document size. In some cases the implementation was close 
enough
of a proxy (PUT-ing and GET-ing single docs), but in some edge cases, like
_bulk_docs requests the discrepancy between request size and document size
could be rather large.

The section was changed accordingly from `couchdb` to `httpd`. `httpd` was
chosen as it applies to both clustered as well as local interface.

There is a parallel effort to implement an actual max_document_size check. 
The
set of commit should be merged close enough together to allow for a 
backwards
compatible transition.

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-couch 
64229-add-new-request-parameter

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-couch/pull/233.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #233


commit d3055f191797f8b637e3b64f6a50e522d2c8bbde
Author: Nick Vatamaniuc 
Date:   2017-03-07T21:47:11Z

Rename max_document_size to max_http_request_size

`max_document_size` is implemented as `max_http_request_size`. There was no
real check for document size. In some cases the implementation was close 
enough
of a proxy (PUT-ing and GET-ing single docs), but in some edge cases, like
_bulk_docs requests the discrepancy between request size and document size
could be rather large.

The section was changed accordingly from `couchdb` to `httpd`. `httpd` was
chosen as it applies to both clustered as well as local interface.

There is a parallel effort to implement an actual max_document_size check. 
The
set of commit should be merged close enough together to allow for a 
backwards
compatible transition.

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900264#comment-15900264
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

Github user asfgit closed the pull request at:

https://github.com/apache/couchdb-couch-replicator/pull/60


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900263#comment-15900263
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 64958096d4f9a940c01cbc472da5265f349c9545 in couchdb-couch-replicator's 
branch refs/heads/master from [~vatamane]
[ 
https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-replicator.git;h=6495809
 ]

Fix unit test after renaming max_document_size config parameter

`couchdb.max_document_size` was renamed to `httpd.max_http_request_size`

The unit tests was testing how replicator behaves when faced with reduced
request size configuration on the target.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900027#comment-15900027
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user nickva opened a pull request:

https://github.com/apache/couchdb-couch-replicator/pull/60

Remove unused mp_parse_doc function from replicator

It was left accidentally when merging Cloudant's dbcore work.

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloudant/couchdb-couch-replicator 
couchdb-2992-remove-dead-code

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-couch-replicator/pull/60.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #60






> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[Nick Vatamaniuc (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15532980#comment-15532980
 ] 

Nick Vatamaniuc commented on COUCHDB-2992:
--

This is related to replications crashing unexpectedly. Users can add documents 
smaller than the limit. Then replicator batches them up to 500 at a time and 
then repeatedly crashes. 

https://issues.apache.org/jira/browse/COUCHDB-3168

> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253408#comment-15253408
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit b0e54930ad341b7385f48d8f06a9455372115e98 in couchdb-chttpd's branch 
refs/heads/2992-limit-doc-size from [~tonysun83]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=b0e5493 ]

Use doc body after parsing

We use the #doc.body because we only care about the json body. Inline
attachments are already stripped away

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253410#comment-15253410
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 0b0a3acaad204d00ea024f15c21dbdf2f01bae38 in couchdb-chttpd's branch 
refs/heads/2992-limit-doc-size from [~tonysun83]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=0b0a3ac ]

Add check for _update handlers

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253409#comment-15253409
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 8ddb1ce7c6bd01511daea3e0d05c5d2edf9610ba in couchdb-chttpd's branch 
refs/heads/2992-limit-doc-size from [~tonysun83]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=8ddb1ce ]

Add in check for body size in multi_part requests

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253407#comment-15253407
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 03fdcde571585f17c2077a3a0313bc8703e32197 in couchdb-chttpd's branch 
refs/heads/2992-limit-doc-size from [~tonysun83]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=03fdcde ]

Move max_document_size logic to update only

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253411#comment-15253411
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit 1ac365dd88d5c953ea739b5b11ebb857038fc7d5 in couchdb-chttpd's branch 
refs/heads/2992-limit-doc-size from [~tonysun83]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=1ac365d ]

Add more tests

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244934#comment-15244934
 ] 

ASF GitHub Bot commented on COUCHDB-2992:
-

GitHub user tonysun83 opened a pull request:

https://github.com/apache/couchdb-chttpd/pull/114

Add support for single_max_doc

Users have been complaining about performance issues as well as indexing 
timeout issues.  Some of these issues are due to large documents. Currently, 
our only limitation is the max_document_size parameter that is set to 64 MB. 
With this enhancement, the goal is to provider some finer grained control over 
single document sizes and provide some more error info. The max_document_size 
still takes precedence. The default max size for a single doc is 1 MB.

During bulk requests, when one of the documents exceeds the 
single_doc_max_size value, it will not be created/updated. The error message 
will simply be "too_large" for that document. Other documents under the limit 
will be created/updated normally.

COUCHDB-2992

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/couchdb-chttpd 2992-limit-doc-size

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/couchdb-chttpd/pull/114.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #114


commit f7affd0d276e12b29e930666dd36ff7b9fdd8e5d
Author: Tony Sun 
Date:   2016-04-17T02:42:46Z

Add support for single_max_doc

This adds support for a new configuration param single_max_doc_size.
Single documents that exceed this document will receive a too_large
error.

COUCHDB-2992




> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (COUCHDB-2992) Add additional support for document size

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244928#comment-15244928
 ] 

ASF subversion and git services commented on COUCHDB-2992:
--

Commit f7affd0d276e12b29e930666dd36ff7b9fdd8e5d in couchdb-chttpd's branch 
refs/heads/2992-limit-doc-size from [~tonysun83]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=f7affd0 ]

Add support for single_max_doc

This adds support for a new configuration param single_max_doc_size.
Single documents that exceed this document will receive a too_large
error.

COUCHDB-2992


> Add additional support for document size
> 
>
> Key: COUCHDB-2992
> URL: https://issues.apache.org/jira/browse/COUCHDB-2992
> Project: CouchDB
>  Issue Type: Improvement
>  Components: Database Core
>Reporter: Tony Sun
>
> Currently, only max_document_size of 64 GB is the restriction for users 
> creating documents. Large documents often leads to issues with our indexers. 
> This feature will allow users more finer grain control over document size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)