Dan Lipofsky created AVRO-2904:
----------------------------------
Summary: timestamp-millis doesn't truncate when in a union with
null
Key: AVRO-2904
URL: https://issues.apache.org/jira/browse/AVRO-2904
Project: Apache Avro
Issue Type: Bug
Components: java, logical types
Affects Versions: 1.10.0
Reporter: Dan Lipofsky
In Avro 1.10.0 the setter for logical-type timestamp-millis will truncate the
timestamp to millis in the simplest condition but not when it is in a union
with null.
It looks like something similar was addressed in AVRO-2360 but that was
supposedly fixed in 1.9.0 and this is still broken in 1.10.0.
Here is my IDL
{noformat}
record TimestampTest1 {
union { null, timestamp_ms } ts;
}
record TimestampTest2 {
timestamp_ms ts;
}
{noformat}
Here is my test, in which {{TimestampTest2}} passes but {{TimestampTest1}}
fails.
{noformat}
public class SerDeTest {
@Test
public void TimestampTest1() throws IOException {
final TimestampTest1 ts =
TimestampTest1.newBuilder().setTs(Instant.now()).build();
byte[] bytes = serialize(ts);
TimestampTest1 other = deserialize(ts.getSchema(), bytes);
Assert.assertEquals(ts, other);
}
@Test
public void TimestampTest2() throws IOException {
final TimestampTest2 ts =
TimestampTest2.newBuilder().setTs(Instant.now()).build();
byte[] bytes = serialize(ts);
TimestampTest2 other = deserialize(ts.getSchema(), bytes);
Assert.assertEquals(ts, other);
}
private <T extends SpecificRecord> T deserialize(Schema schema, byte[]
bytes)
throws IOException {
Decoder decoder = DecoderFactory.get().binaryDecoder(bytes, null);
SpecificDatumReader<T> reader = new SpecificDatumReader<>(schema);
return reader.read(null, decoder);
}
private byte[] serialize(SpecificRecord record) throws IOException {
try (final ByteArrayOutputStream out = new ByteArrayOutputStream()) {
SpecificDatumWriter<SpecificRecord> writer = new
SpecificDatumWriter<>(
record.getSchema());
Encoder encoder = EncoderFactory.get().binaryEncoder(out, null);
writer.write(record, encoder);
encoder.flush();
return out.toByteArray();
}
}
}
{noformat}
{{TimestampTest1}} fails with
{noformat}
java.lang.AssertionError:
Expected :{"ts": 2020-07-23T21:35:12.348379Z}
Actual :{"ts": 2020-07-23T21:35:12.348Z}
{noformat}
If I change my tests to use {{Instant.now().truncatedTo(ChronoUnit.MILLIS)}}
they'll both pass, but it seems like that should not be necessary, particularly
given the code below.
Looking at the generated Java DTO, I see a few differences, but the most
important would seem to be that the working DTO has
{noformat}
public void setTs(java.time.Instant value) {
this.ts = value.truncatedTo(java.time.temporal.ChronoUnit.MILLIS);
}
{noformat}
while the broken one has
{noformat}
public void setTs(java.time.Instant value) {
this.ts = value;
}
{noformat}
The working one also has this code, which is missing from the broken one:
{noformat}
private static final org.apache.avro.Conversion<?>[] conversions =
new org.apache.avro.Conversion<?>[] {
new org.apache.avro.data.TimeConversions.TimestampMillisConversion(),
null
};
@Override
public org.apache.avro.Conversion<?> getConversion(int field) {
return conversions[field];
}
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
