Dear Parquet Experts,
I tried running some parquet-tools commands using version 1.9.1 and version
1.6.1 and was quite surprised to see that the newer parquet-tools doesn't
work for me at all:
$ java -jar parquet-tools-*1.6.1*-SNAPSHOT.jar schema /tmp/test.par
message hive_schema {
optional int32 a;
optional binary b (UTF8);
}
$ java -jar parquet-tools-*1.9.1*-SNAPSHOT.jar schema /tmp/test.par
Unknown command: schema
$ java -jar parquet-tools-*1.6.1*-SNAPSHOT.jar meta /tmp/test.par
file: file:/tmp/test.par
creator: parquet-mr version 1.5.0-cdh5.10.0-SNAPSHOT (build
${buildNumber})
file schema: hive_schema
--------------------------------------------------------------------------------
a: OPTIONAL INT32 R:0 D:1
b: OPTIONAL BINARY O:UTF8 R:0 D:1
row group 1: RC:2 TS:90 OFFSET:4
--------------------------------------------------------------------------------
a: INT32 UNCOMPRESSED DO:0 FPO:4 SZ:47/47/1,00 VC:2
ENC:PLAIN,RLE,BIT_PACKED
b: BINARY UNCOMPRESSED DO:0 FPO:51 SZ:43/43/1,00 VC:2
ENC:PLAIN,RLE,BIT_PACKED
$ java -jar parquet-tools-*1.9.1*-SNAPSHOT.jar meta /tmp/test.par
org/apache/hadoop/conf/Configuration
$ java -jar parquet-tools-*1.6.1*-SNAPSHOT.jar cat /tmp/test.par
a = 0
b = a
a = 1
b = b
$ java -jar parquet-tools-*1.9.1*-SNAPSHOT.jar cat /tmp/test.par
org/apache/hadoop/fs/Path
I tried version 1.8.1 as well, which behaves similar to 1.9.1. The output
of the meta and cat commands just seems to be plain bad, but the error
message for the schema command suggests that there was some intentional
change in parquet commands. I tried to find out what happened to the schema
command, but I didn't find anything. In fact, it seems to me that the
schema command should still be present
<https://github.com/apache/parquet-mr/blob/master/parquet-tools/src/main/java/org/apache/parquet/tools/command/Registry.java#L31>
.
Can anyone tell me whether the behavior I experience is the same as for
others and if so, is it intentional?
Thanks,
Zoltan
