[
https://issues.apache.org/jira/browse/BEAM-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059968#comment-16059968
]
ASF GitHub Bot commented on BEAM-2439:
--------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/3390
> Datastore writer can fail to progress if Datastore is slow
> ----------------------------------------------------------
>
> Key: BEAM-2439
> URL: https://issues.apache.org/jira/browse/BEAM-2439
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-gcp
> Reporter: Colin Phipps
> Assignee: Stephen Sisk
> Priority: Minor
> Labels: datastore
>
> When writing to Datastore, Beam groups writes into large batches (usually 500
> entities per write, the maximum permitted by the API). If these writes are
> slow to commit on the serving side, the request may time out before all of
> the entities are written.
> When this happens, it loses any progress that has been made on those entities
> (the connector uses non-transactional writes, so some entities might have
> been written, but partial results are not returned to the connector so it has
> to assume that all entities need rewriting). It will retry the write with the
> same set of entities, which may time out in the same way repeatedly. This can
> be influenced by factors on the Datastore serving side, some of which are
> transient (hotspots) but some of which are not.
> We (Datastore) are developing a fix for this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
