[
https://issues.apache.org/jira/browse/BEAM-11403?focusedWorklogId=527764&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527764
]
ASF GitHub Bot logged work on BEAM-11403:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Dec/20 18:48
Start Date: 23/Dec/20 18:48
Worklog Time Spent: 10m
Work Description: boyuanzz commented on pull request #13592:
URL: https://github.com/apache/beam/pull/13592#issuecomment-750427257
> Great, I think this covers pretty much all the topics. I have only one
small concern left - because the key in cache is derived from structuralValue
of RestrictionCoder, it uses SerializableCoder for encoding the source. That
might not be 100% stable. I would prefer to generate a different identifier in
`UnboundedSourceAsSDFWrapperFn.splitRestriction` for each split and then carry
it along all calls to `trySplit`. But this might be viewed as minor concern.
I would not worry about that too much since we are using `Source` +
`CheckpointMark` as the restriction, as well as the cache key.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 527764)
Time Spent: 3h (was: 2h 50m)
> Unbounded SDF wrapper causes performance regression on DirectRunner
> -------------------------------------------------------------------
>
> Key: BEAM-11403
> URL: https://issues.apache.org/jira/browse/BEAM-11403
> Project: Beam
> Issue Type: Bug
> Components: runner-direct, sdk-java-core
> Reporter: Boyuan Zhang
> Assignee: Boyuan Zhang
> Priority: P2
> Time Spent: 3h
> Remaining Estimate: 0h
>
> There is a significant performance regression when switching from
> UnboundedSource to Unbounded SDF wrapper. So far there are 2 IOs reported:
> * Pubsub Read:
> https://lists.apache.org/thread.html/re6b0941a8b4951293a0327ce9b25e607cafd6e45b69783f65290edee%40%3Cdev.beam.apache.org%3E
> * Kafka Read: https://the-asf.slack.com/archives/C9H0YNP3P/p1606155042346600
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
