[
https://issues.apache.org/jira/browse/BEAM-12883?focusedWorklogId=663237&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663237
]
ASF GitHub Bot logged work on BEAM-12883:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Oct/21 08:45
Start Date: 10/Oct/21 08:45
Worklog Time Spent: 10m
Work Description: brachi-wernick opened a new pull request #15699:
URL: https://github.com/apache/beam/pull/15699
This PR is a continuous work for https://github.com/apache/beam/pull/15510.
Currently there are 2 Coders for Metadata: default one:
`org.apache.beam.sdk.io.fs.MetadataCoder` and enhanced one
`org.apache.beam.sdk.io.fs.MetadataCoderV2`, the last can also decode-encode
`lastModifiedMillis` and it is done in a new coder in order to support backward
compatibility.
This will be hard to maintain, we will need to create a new coder for any
new field that will be added to `Metadata`.
So, as suggested in this comment:
https://github.com/apache/beam/pull/15510#issuecomment-928390587, I came up
with new generic coder : `MetadataDynamicCoder`.
`MetadataDynamicCoder` can decode/encode any new fields added to `Metadata`
by sending getter, setter and coder.
For example creating coder for `lastModifiedMillis`:
``` java
new MetadataDynamicCoder()
.withCoderForField(
VarLongCoder.of(),
Metadata::lastModifiedMillis,
Metadata.Builder::setLastModifiedMillis);
```
I chose to get explicit getter/setter to avoid reflection which has bad
impact on performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 663237)
Time Spent: 2h 50m (was: 2h 40m)
> FileSystems ReadableFileCoder should support Metadata#lastModifiedMillis()
> --------------------------------------------------------------------------
>
> Key: BEAM-12883
> URL: https://issues.apache.org/jira/browse/BEAM-12883
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Brachi Packter
> Priority: P3
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Currently, there is a separated coder for Metadata which cares about
> `Metadata#lastModifiedMillis()`.
> The issue is that the `ReadableFileCoder` always uses the default `Metadata`
> coder.
> We need to create a similar coder for `ReadableFileCoder`:
> ``ReadableFileCoderV2` which just use the corresponding `MetadataCoderV2`.
> Without doing so `lastModifiedMillis` will remain zero.
> There is also no way to create custom coder in our side since
> `FileIO.ReadableFile` has no public method to initiate new instance of it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
