alippai commented on code in PR #34: URL: https://github.com/apache/parquet-site/pull/34#discussion_r1237281423
########## content/en/docs/File Format/implementationstatus.md: ########## @@ -0,0 +1,178 @@ +--- +title: "Implementation status" +linkTitle: "Implementation status" +weight: 8 +--- + +### Physical types + ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Data type | C++ | Python | Java | Go | Rust | +| | | | | | | ++===========================================+=======+========+========+=======+=======+ +| BOOLEAN | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| INT32 | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| INT64 | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| INT96 | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| FLOAT | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DOUBLE | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| BYTE_ARRAY | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| FIXED_LEN_BYTE_ARRAY | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ + +### Logical types + ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Data type | C++ | Python | Java | Go | Rust | +| | | | | | | ++===========================================+=======+========+========+=======+=======+ +| STRING | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| ENUM | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| UUID | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| 8 and 16 bit signed INT | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| 8, 16, 32, 64 bit unsigned INT | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DECIMAL (INT32) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DECIMAL (INT64) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DECIMAL (BYTE_ARRAY) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DECIMAL (FIXED_LEN_BYTE_ARRAY) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DATE | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| TIME (INT32) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| TIME (INT64) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| TIMESTAMP (INT32) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| TIMESTAMP (INT64) | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| INTERVAL | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| JSON | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| BSON | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| LIST | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| MAP | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| UNKNOWN | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ + +### Encoding + ++-------------------------------------------+-------+--------+--------+-------+-------+ +| | C++ | Python | Java | Go | Rust | +| | | | | | | ++===========================================+=======+========+========+=======+=======+ +| PLAIN | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| PLAIN_DICTIONARY | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| RLE_DICTIONARY | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| RLE | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| BIT_PACKED | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DELTA_BINARY_PACKED | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DELTA_LENGTH_BYTE_ARRAY | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| DELTA_BYTE_ARRAY | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| BYTE_STREAM_SPLIT | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ + +### Compression + ++-------------------------------------------+-------+--------+--------+-------+-------+ +| | C++ | Python | Java | Go | Rust | +| | | | | | | ++===========================================+=======+========+========+=======+=======+ +| UNCOMPRESSED | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| SNAPPY | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| GZIP | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| LZO | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| BROTLI | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| LZ4 | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| ZSTD | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| LZ4_RAW | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ + +### Other format level features + ++-------------------------------------------+-------+--------+--------+-------+-------+ +| | C++ | Python | Java | Go | Rust | +| | | | | | | ++===========================================+=======+========+========+=======+=======+ +| xxHash Bloom filters | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| bloom filter length | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Statistics min_value, max_value | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Column index | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Offset index | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Modular encryption | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Page CRC32 checksum | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ +| Modular encryption | | | | | | ++-------------------------------------------+-------+--------+--------+-------+-------+ + +High level data API-s for parquet feature usage +=============================================== + ++----------------------------------------------+-------+--------+--------+-------+-------+ +| Format | C++ | Python | Java | Go | Rust | +| | | | | | | ++==============================================+=======+========+========+=======+=======+ +| Hive-style partitioning | | | | | | ++----------------------------------------------+-------+--------+--------+-------+-------+ +| Partition pruning on the partition column | | | | | | ++----------------------------------------------+-------+--------+--------+-------+-------+ +| External column data | | | | | | Review Comment: Will do -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
