[
https://issues.apache.org/jira/browse/BEAM-11249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253104#comment-17253104
]
Brian Hulette commented on BEAM-11249:
--------------------------------------
Filed BEAM-11505 to track dynamically setting chunksize
> Read a reasonable amount of data per chunk.
> -------------------------------------------
>
> Key: BEAM-11249
> URL: https://issues.apache.org/jira/browse/BEAM-11249
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Robert Bradshaw
> Assignee: Robert Bradshaw
> Priority: P2
> Fix For: 2.26.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In read_csv (and others), we are reading only 100 rows or so at a time. This
> gives kb-sized chunks, the reader is likely optimal in the mb-sized ranges.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
