[
https://issues.apache.org/jira/browse/BEAM-9960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17548591#comment-17548591
]
Danny McCormick commented on BEAM-9960:
---------------------------------------
This issue has been migrated to https://github.com/apache/beam/issues/20215
> Python MongoDBIO fails when response of split vector command is larger than
> 16mb
> --------------------------------------------------------------------------------
>
> Key: BEAM-9960
> URL: https://issues.apache.org/jira/browse/BEAM-9960
> Project: Beam
> Issue Type: Bug
> Components: io-py-mongodb
> Affects Versions: 2.20.0
> Reporter: Corvin Deboeser
> Priority: P3
>
> When using MongoDBIO on a large collection with large documents on average,
> then the split vector command results in a lot of splits if the desired
> bundle size is small. In extreme cases, the response from the split vector
> command can be larger than 16mb which is not supported by pymongo / MongoDB:
> {{pymongo.errors.ProtocolError: Message length (33699186) is larger than
> server max message size (33554432)}}
>
> Environment: Was running this on Google Dataflow / Beam Python SDK 2.20.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
