[
https://issues.apache.org/jira/browse/PARQUET-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806054#comment-15806054
]
Ryan Blue commented on PARQUET-725:
-----------------------------------
Another work-around for this is to use slf4j-simple during tests, which is
configured at INFO by the root POM. The reason why the log level is DEBUG is
that slf4j-log4j12 is being pulled in transitively through hadoop-client.
Excluding that dependency (which is provided anyway) avoids the problem. We
still need to update Avro once it is fixed, but this will allow builds to pass
for now.
{code:lang=xml}
diff --git a/parquet-avro/pom.xml b/parquet-avro/pom.xml
index b34838f..5eaf1e0 100644
--- a/parquet-avro/pom.xml
+++ b/parquet-avro/pom.xml
@@ -63,6 +63,13 @@
<artifactId>hadoop-client</artifactId>
<version>${hadoop.version}</version>
<scope>provided</scope>
+ <exclusions>
+ <exclusion>
+ <!-- this changes logger used in testing -->
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
+ </exclusions>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
{code}
> Parquet AVRO tests fail when debug logging is enabled
> -----------------------------------------------------
>
> Key: PARQUET-725
> URL: https://issues.apache.org/jira/browse/PARQUET-725
> Project: Parquet
> Issue Type: Bug
> Reporter: Niels Basjes
> Assignee: Niels Basjes
>
> I found that on my machine some of the tests in the parquet-avro fail.
> {code}
> Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.073 sec
> Running org.apache.parquet.avro.TestAvroDataSupplier
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> Running org.apache.parquet.avro.TestReadWrite
> Tests run: 18, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 0.414 sec
> <<< FAILURE!
> Running org.apache.parquet.avro.TestBackwardCompatibility
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
> Running org.apache.parquet.avro.TestReadWriteOldListBehavior
> Tests run: 16, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 0.148 sec
> <<< FAILURE!
> Running org.apache.parquet.avro.TestInputOutputFormat
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.29 sec
> Running org.apache.parquet.avro.TestReflectLogicalTypes
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.165 sec
> Running org.apache.parquet.avro.TestCircularReferences
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> Results :
> Failed tests:
> testWriteReflectReadGeneric(org.apache.parquet.avro.TestReflectReadWrite):
> expected:<{"myboolean": true, "mybyte": 1, "myshort": 1, "myint": 1,
> "mylong": 2, "myfloat": 3.1, "mydouble": 4.1, "mybytes": {"bytes":
> "\u0001\u0002\u0003\u0004"}, "mystring": "Hello", "myenum": "A", "mymap":
> {"a": "1", "b": "2"}, "myshortarray": [1, 2], "myintarray": [1, 2],
> "mystringarray": ["a", "b"], "mylist": ["a", "b", "c"]}> but
> was:<{"myboolean": true, "mybyte": 1, "myshort": 1, "myint": 1, "mylong": 2,
> "myfloat": 3.1, "mydouble": 4.1, "mybytes": {"bytes": ""}, "mystring":
> "Hello", "myenum": "A", "mymap": {"a": "1", "b": "2"}, "myshortarray": [1,
> 2], "myintarray": [1, 2], "mystringarray": ["a", "b"], "mylist": ["a", "b",
> "c"]}>
> testWriteDecimalBytes(org.apache.parquet.avro.TestGenericLogicalTypes):
> Should read BigDecimals as bytes expected:<[{"dec": {"bytes": "ò\u0096"}},
> {"dec": {"bytes": "\u0000²àø"}}]> but was:<[{"dec": {"bytes": ""}}, {"dec":
> {"bytes": ""}}]>
> testAll[0](org.apache.parquet.avro.TestReadWrite):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
> testAllUsingDefaultAvroSchema[0](org.apache.parquet.avro.TestReadWrite):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
> testAll[1](org.apache.parquet.avro.TestReadWrite):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
> testAllUsingDefaultAvroSchema[1](org.apache.parquet.avro.TestReadWrite):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
> testAll[0](org.apache.parquet.avro.TestReadWriteOldListBehavior):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
>
> testAllUsingDefaultAvroSchema[0](org.apache.parquet.avro.TestReadWriteOldListBehavior):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
> testAll[1](org.apache.parquet.avro.TestReadWriteOldListBehavior):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
>
> testAllUsingDefaultAvroSchema[1](org.apache.parquet.avro.TestReadWriteOldListBehavior):
> expected:<java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
> {code}
> I see two classes of problems:
> # The json with byte arrays appear different.
> # Some tests compare the 'toString' of a ByteBuffer. Now for two ByteBuffers
> that both contain the SAME bytes these tests fail simply because the position
> field of the ByteBuffer is different. I think these should compare the
> contents of the ByteBuffer instead.
> {code}
> <java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]> but
> was:<java.nio.HeapByteBuffer[pos=5 lim=5 cap=5]>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
