[ 
https://issues.apache.org/jira/browse/FLINK-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573133#comment-15573133
 ] 

ASF GitHub Bot commented on FLINK-4771:
---------------------------------------

Github user qingdao81 commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2612#discussion_r83307880
  
    --- Diff: 
flink-batch-connectors/flink-avro/src/test/java/org/apache/flink/api/java/io/AvroOutputFormatTest.java
 ---
    @@ -0,0 +1,126 @@
    +package org.apache.flink.api.java.io;
    +
    +import static org.apache.flink.api.java.io.AvroOutputFormat.Codec;
    +import static org.junit.Assert.assertTrue;
    +import static org.junit.Assert.fail;
    +
    +import java.io.File;
    +import java.io.FileInputStream;
    +import java.io.FileOutputStream;
    +import java.io.IOException;
    +import java.io.ObjectInputStream;
    +import java.io.ObjectOutputStream;
    +import java.nio.file.Files;
    +import java.nio.file.Paths;
    +
    +import org.apache.flink.api.io.avro.example.User;
    +import org.apache.flink.configuration.Configuration;
    +import org.apache.flink.core.fs.FileSystem;
    +import org.apache.flink.core.fs.Path;
    +import org.junit.Test;
    +
    +/**
    + * Tests for {@link AvroOutputFormat}
    + */
    +public class AvroOutputFormatTest {
    +
    +    @Test
    +    public void testSetCodec() throws Exception {
    +        // given
    +        final AvroOutputFormat<User> outputFormat = new 
AvroOutputFormat<>(User.class);
    +
    +        // when
    +        try {
    +            outputFormat.setCodec(Codec.SNAPPY);
    +        } catch (Exception ex) {
    +            // then
    +            fail("unexpected exception");
    +        }
    +    }
    +
    +    @Test
    +    public void testSetCodecError() throws Exception {
    +        // given
    +        boolean error = false;
    +        final AvroOutputFormat<User> outputFormat = new 
AvroOutputFormat<>(User.class);
    +
    +        // when
    +        try {
    +            outputFormat.setCodec(null);
    +        } catch (Exception ex) {
    +            error = true;
    +        }
    +
    +        // then
    +        assertTrue(error);
    +    }
    +
    +    @Test
    +    public void testSerialization() throws Exception {
    +
    +        serializeAndDeserialize(null);
    +        for (final Codec codec : Codec.values()) {
    +            serializeAndDeserialize(codec);
    +        }
    +    }
    +
    +    private void serializeAndDeserialize(final Codec codec) throws 
IOException, ClassNotFoundException {
    +        // given
    +        final AvroOutputFormat<User> outputFormat = new 
AvroOutputFormat<>(User.class);
    +        if (codec != null) {
    +            outputFormat.setCodec(codec);
    +        }
    +        outputFormat.setSchema(User.SCHEMA$);
    +
    +        final File serialized = File.createTempFile("avro-output-format", 
".serialized");
    +
    +        // when
    +        try (final ObjectOutputStream oos = new ObjectOutputStream(new 
FileOutputStream(serialized))) {
    +            oos.writeObject(outputFormat);
    +        }
    +        try (final ObjectInputStream ois = new ObjectInputStream(new 
FileInputStream(serialized))) {
    +            // then
    +            assertTrue(ois.readObject() instanceof AvroOutputFormat);
    --- End diff --
    
    You are right this should be checked but I'm not sure what is the best way 
to do so ... I could change the visibility for userDefinedSchema and codec to 
package private or provide a getter for both properties.
    But I would increase the visibility of params just for test purposes. Some 
people are ok with this others not ;-) or do you have something else in mind? - 
I could also write a d read some data after the serialization to verify schema 
and compression but in my opinion this would be out of the scope of a 
serialization test.


> Compression for AvroOutputFormat
> --------------------------------
>
>                 Key: FLINK-4771
>                 URL: https://issues.apache.org/jira/browse/FLINK-4771
>             Project: Flink
>          Issue Type: Improvement
>          Components: Batch Connectors and Input/Output Formats
>            Reporter: Lars Bachmann
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Currently it is not possible to set a compression codec for the 
> AvroOutputFormat. 
> This improvement will provide a setter for the avro CodecFactory which is 
> used by the DataFileWriter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to