[
https://issues.apache.org/jira/browse/BEAM-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809952#comment-16809952
]
Kyle Winkelman commented on BEAM-6713:
--------------------------------------
At this time we don't want to expose an additional knob. Look at
{BEAM-6735|https://issues.apache.org/jira/browse/BEAM-6735] as a possible
alternative to remove the limit imposed by DEFAULT_MAX_NUM_WRITERS_PER_BUNDLE.
> FileIO and TextIO unable to alter WriteFiles maxNumWritersPerBundle
> -------------------------------------------------------------------
>
> Key: BEAM-6713
> URL: https://issues.apache.org/jira/browse/BEAM-6713
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Kyle Winkelman
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When attempting to run a batch workflow with a FileIO.write() I was getting
> job failures due to WriteFiles.DEFAULT_MAX_NUM_WRITERS_PER_BUNDLE causing a
> significant amount of data to be shuffled. My issues would be solved by
> increasing this and luckily WriteFiles already has withMaxNumWritersPerBundle
> but unfortunately FileIO and TextIO do not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
