[
https://issues.apache.org/jira/browse/BEAM-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072582#comment-16072582
]
Reuven Lax commented on BEAM-2353:
----------------------------------
In that case we should take https://github.com/apache/beam/pull/3356 as
well, as that also is a breaking change to the FilenamePolicy interface
(removing baseDirectory and extension). Last word from Eugene was that he
was happy with it but was going on vacation. He told me to relay the
message that he was happy for any other committer to merge the PR.
An Apex runner test currently breaks with this PR. As far as I can tell
debugging, this is simply a bug in the Apex runner. The root cause appears
to be the Apex runner throwing some sort of socket exception when one of
the ParDos calls output. I've started a thread on the dev list about this,
but IMO should not be a blocker for this PR.
On Tue, May 23, 2017 at 2:07 PM, Kenneth Knowles (JIRA) <[email protected]>
> FileNamePolicy context parameters allow backwards compatibility where we
> really don't want any
> ----------------------------------------------------------------------------------------------
>
> Key: BEAM-2353
> URL: https://issues.apache.org/jira/browse/BEAM-2353
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Kenneth Knowles
> Assignee: Reuven Lax
> Fix For: 2.2.0
>
>
> Currently, in {{FileBasedSink}} the {{FileNamePolicy}} object accepts
> parameters of type {{Context}} and {{WindowedContext}} respectively.
> These contexts are a coding technique to allow easy backwards compatibility
> when adding new parameters. However, if a new parameter is added to the file
> name policy it is likely data loss for the user to not incorporate it, so in
> fact that is never a safe backwards compatible change.
> These are brand-new APIs and marked experimental. This is important enough I
> think we should make the breaking change.
> We should inline all the parameters of the context, so that we _cannot_ add
> parameters and maintain compatibility. Instead, if we have new ones we want
> to add, it will have to be a new method or some such.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
