sheepdreamofandroids commented on issue #22840: URL: https://github.com/apache/beam/issues/22840#issuecomment-1226095863
Hmmm, I didn't think about flattening into a single BulkIO. That would indeed solve the multiple batches problem. It still makes the code more complicated than necessary because what could be a simple switch statement now has to be a dispatch to multiple DocToBulks. My idea was to write my own DocToBulk with the necessary logic. It just means that I would have to duplicate `createBulkApiEntity` and `getDocumentMetadata` which isn't too complicated. Would you accept a PR with a type parameter on DocToBulk and additional jsonFn and upsertFn? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
