[
https://issues.apache.org/jira/browse/BEAM-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757435#comment-16757435
]
Jeff Klukas commented on BEAM-4822:
-----------------------------------
Does schema-based coding handle evolution of types? I'm wondering if the
automatic schema generation we now have for AutoValue classes would allow us to
handle this evolution more elegantly.
If a record is encoded with a schema-based coder with a previous version of
Beam, and then we add a new field, is the schema-based coder able to decode
into the class with the added field? Perhaps the field would have to be
nullable?
If so, we could add an AutoValue-based MetadataAdditionalProperties that could
serve as the bag for adding additional metadata in the future.
We can investigate what using a Map<String, String> would look like, but I
think it's going to be messy and will need significant documentation to help
users understand what keys to expect.
> Beam FileIO should support versioned file systems
> -------------------------------------------------
>
> Key: BEAM-4822
> URL: https://issues.apache.org/jira/browse/BEAM-4822
> Project: Beam
> Issue Type: New Feature
> Components: io-java-files
> Reporter: Reuven Lax
> Assignee: Chamikara Jayalath
> Priority: Major
>
> Some file systems (e.g. GCS) are versioned, and support reading previous
> generations of files. Since Beam's file support does not currently support
> this concept, the latest versions of files are always the ones returned.
> Users should be able to specify that they want to read a previous version of
> a file in FileIO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
