Hello, I'm relatively new to parquet and to the community, kindly bear with me if I'm making simple mistakes.
I cloned parquet-mr and built the repo locally on my ubuntu 20.04, When running parquet-cli, particularly when trying to read values (cat, head) from a parquet file, I run into NoSuchMethodError (full stack at the end of message). *Here is what I have tried so far to understand this problem:* 1. clean the repo and clone again and retry 2. metadata inspection commands such as `meta`, `schema`, `pages` `dictionary` - they all work perfectly. check-stats command reports no corruption or errors, 3. I'm running parquet-cli without Hadoop (according to this ReadMe <https://github.com/apache/parquet-mr/tree/master/parquet-cli#running-without-hadoop> ) 4. I have tried this on two different machines (ubuntu 18.04 and ubuntu 20.04) and gotten the same result. 5. The input file is produced by a library that I am writing to serialize parquet data in .net (i.e, not produced by parquet-mr). 6. parquet-cpp is able to read the file correctly and deserialize the data. parquet-mr compatibility is important to me, so I'm trying to get this working. I could not solve this problem, so, I would really appreciate it if someone who understands what's wrong here could kindly share insights. *Error stack:* ~/repos/parquet-mr/parquet-cli$ parquet cat ../../testdata/dictionaryEncodingSample.parquet WARNING: An illegal reflective access operation has occurred ...<trimmed some more WARNs>... Exception in thread "main" java.lang.NoSuchMethodError: 'org.apache.avro.Schema org.apache.parquet.avro.AvroSchemaConverter.convert(org.apache.parquet.schema.MessageType)' at org.apache.parquet.cli.util.Schemas.fromParquet(Schemas.java:89) at org.apache.parquet.cli.BaseCommand.getAvroSchema(BaseCommand.java:405) at org.apache.parquet.cli.commands.CatCommand.run(CatCommand.java:66) at org.apache.parquet.cli.Main.run(Main.java:157) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.parquet.cli.Main.main(Main.java:187) *mvn dependency:tree output for parquet-cli* [INFO] -------------------< org.apache.parquet:parquet-cli >------------------- [INFO] Building Apache Parquet Command-line 1.13.0-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ parquet-cli --- [INFO] org.apache.parquet:parquet-cli:jar:1.13.0-SNAPSHOT [INFO] +- org.apache.parquet:parquet-avro:jar:1.13.0-SNAPSHOT:compile [INFO] +- org.apache.parquet:parquet-format-structures:jar:1.13.0-SNAPSHOT:compile [INFO] +- org.apache.parquet:parquet-common:jar:1.13.0-SNAPSHOT:compile [INFO] +- org.apache.parquet:parquet-column:jar:1.13.0-SNAPSHOT:compile [INFO] | +- org.apache.parquet:parquet-encoding:jar:1.13.0-SNAPSHOT:compile [INFO] | \- org.apache.yetus:audience-annotations:jar:0.13.0:compile [INFO] +- org.apache.parquet:parquet-hadoop:jar:1.13.0-SNAPSHOT:compile [INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.8.3:compile [INFO] | \- commons-pool:commons-pool:jar:1.6:compile [INFO] +- org.apache.avro:avro:jar:1.10.2:compile [INFO] | \- org.apache.commons:commons-compress:jar:1.20:compile [INFO] +- com.github.luben:zstd-jni:jar:1.5.0-1:runtime [INFO] +- org.slf4j:slf4j-api:jar:1.7.22:compile [INFO] +- net.sf.opencsv:opencsv:jar:2.3:compile [INFO] +- org.apache.commons:commons-text:jar:1.8:compile [INFO] | \- org.apache.commons:commons-lang3:jar:3.9:compile [INFO] +- org.apache.parquet:parquet-jackson:jar:1.13.0-SNAPSHOT:runtime [INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.12.2:compile [INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.12.2:compile [INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.2:compile [INFO] +- com.beust:jcommander:jar:1.72:compile [INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.22:provided [INFO] +- com.google.guava:guava:jar:27.0.1-jre:provided [INFO] | +- com.google.guava:failureaccess:jar:1.0.1:provided [INFO] | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:provided [INFO] | +- org.checkerframework:checker-qual:jar:2.5.2:provided [INFO] | +- com.google.errorprone:error_prone_annotations:jar:2.2.0:provided [INFO] | +- com.google.j2objc:j2objc-annotations:jar:1.1:provided [INFO] | \- org.codehaus.mojo:animal-sniffer-annotations:jar:1.17:provided [INFO] +- org.apache.hadoop:hadoop-client:jar:2.10.1:provided [INFO] | +- org.apache.hadoop:hadoop-hdfs-client:jar:2.10.1:provided [INFO] | | \- com.squareup.okhttp:okhttp:jar:2.7.5:provided [INFO] | | \- com.squareup.okio:okio:jar:1.6.0:provided [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.10.1:provided [INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.10.1:provided [INFO] | | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.10.1:provided [INFO] | | +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.10.1:provided [INFO] | | | +- org.apache.hadoop:hadoop-yarn-registry:jar:2.10.1:provided [INFO] | | | +- org.apache.geronimo.specs:geronimo-jcache_1.0_spec:jar:1.0-alpha-1:provided [INFO] | | | +- org.ehcache:ehcache:jar:3.3.1:provided [INFO] | | | +- com.zaxxer:HikariCP-java7:jar:2.4.12:provided [INFO] | | | \- com.microsoft.sqlserver:mssql-jdbc:jar:6.2.1.jre7:provided [INFO] | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:provided [INFO] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.10.1:provided [INFO] | | \- javax.xml.bind:jaxb-api:jar:2.2.2:provided [INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:provided [INFO] | | \- javax.activation:activation:jar:1.1:provided [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.10.1:provided [INFO] | | +- org.apache.hadoop:hadoop-yarn-client:jar:2.10.1:provided [INFO] | | \- org.apache.hadoop:hadoop-yarn-common:jar:2.10.1:provided [INFO] | | \- com.sun.jersey:jersey-client:jar:1.9:provided [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.10.1:provided [INFO] | \- org.apache.hadoop:hadoop-annotations:jar:2.10.1:provided [INFO] +- org.apache.hadoop:hadoop-common:jar:2.10.1:provided [INFO] | +- commons-cli:commons-cli:jar:1.2:provided [INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:provided [INFO] | +- xmlenc:xmlenc:jar:0.52:provided [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.2:provided [INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.4:provided [INFO] | +- commons-codec:commons-codec:jar:1.4:provided [INFO] | +- commons-net:commons-net:jar:3.1:provided [INFO] | +- commons-collections:commons-collections:jar:3.2.2:provided [INFO] | +- javax.servlet:servlet-api:jar:2.5:provided [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:provided [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:provided [INFO] | +- org.mortbay.jetty:jetty-sslengine:jar:6.1.26:provided [INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:provided [INFO] | +- com.sun.jersey:jersey-core:jar:1.9:provided [INFO] | +- com.sun.jersey:jersey-json:jar:1.9:provided [INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:provided [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:provided [INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:provided [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:provided [INFO] | +- com.sun.jersey:jersey-server:jar:1.9:provided [INFO] | | \- asm:asm:jar:3.1:provided [INFO] | +- net.java.dev.jets3t:jets3t:jar:0.9.0:provided [INFO] | | \- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:provided [INFO] | +- commons-lang:commons-lang:jar:2.6:provided [INFO] | +- commons-configuration:commons-configuration:jar:1.6:provided [INFO] | +- commons-digester:commons-digester:jar:1.8:provided [INFO] | +- commons-beanutils:commons-beanutils:jar:1.9.4:provided [INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided [INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided [INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:provided [INFO] | +- com.google.code.gson:gson:jar:2.2.4:provided [INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.10.1:provided [INFO] | | +- com.nimbusds:nimbus-jose-jwt:jar:7.9:provided [INFO] | | | +- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:provided [INFO] | | | \- net.minidev:json-smart:jar:2.3:provided (version selected from constraint [1.3.1,2.3]) [INFO] | | | \- net.minidev:accessors-smart:jar:1.2:provided [INFO] | | | \- org.ow2.asm:asm:jar:5.0.4:provided [INFO] | | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:provided [INFO] | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:provided [INFO] | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:provided [INFO] | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:provided [INFO] | | \- org.apache.curator:curator-framework:jar:2.13.0:provided [INFO] | +- com.jcraft:jsch:jar:0.1.55:provided [INFO] | +- org.apache.curator:curator-client:jar:2.13.0:provided [INFO] | +- org.apache.curator:curator-recipes:jar:2.13.0:provided [INFO] | +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:provided [INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.14:provided [INFO] | | +- com.github.spotbugs:spotbugs-annotations:jar:3.1.9:provided [INFO] | | \- io.netty:netty:jar:3.10.6.Final:provided [INFO] | +- org.codehaus.woodstox:stax2-api:jar:3.1.4:provided [INFO] | \- com.fasterxml.woodstox:woodstox-core:jar:5.0.3:provided [INFO] +- com.google.code.findbugs:jsr305:jar:3.0.2:provided [INFO] +- log4j:log4j:jar:1.2.17:provided [INFO] +- commons-io:commons-io:jar:2.4:provided [INFO] +- commons-logging:commons-logging:jar:1.1.3:provided [INFO] +- junit:junit:jar:4.13.1:test [INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test [INFO] \- org.easymock:easymock:jar:3.4:test [INFO] \- org.objenesis:objenesis:jar:2.2:test [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ Thank you
