GANHONGNAN created PARQUET-2340:
-----------------------------------
Summary: appendRowGroup support append pageIndex
Key: PARQUET-2340
URL: https://issues.apache.org/jira/browse/PARQUET-2340
Project: Parquet
Issue Type: Improvement
Components: parquet-mr
Reporter: GANHONGNAN
Currently,
org.apache.parquet.hadoop.ParquetFileWriter#appendFile(org.apache.parquet.io.InputFile)
uses appendRowGroup method to concate parquet row group. However,
appendRowGroup method *looses* column index.
{code:java}
// code placeholder
public void appendRowGroup(SeekableInputStream from, BlockMetaData rowGroup,
boolean dropColumns) throws IOException {
// TODO: column/offset indexes are not copied
// (it would require seeking to the end of the file for each row groups)
currentColumnIndexes.add(null);
currentOffsetIndexes.add(null);
} {code}
[https://github.com/apache/parquet-mr/blob/f8465a274b42e0a96996c76f3be0b50cf85ecf15/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java#L1033C19-L1033C19]
Look forward to functionality that support append with page index.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
