I don't think this is possible due to the encoding and compression schemes.
For example, suppose that you had the following data 1 1 1 4 4 4 4 This would be dictionary-encoded and compressed to semantically look like dictionary: 1, 4 data page: (3, 0) (4, 1) The encoded data page (using the hybrid bit-packing / RLE encoding scheme) would furthermore be compressed. Editing records in general would change the size of the compressed and encoded data stream, so you could not edit the page without rewriting the file. - Wes On Mon, Dec 4, 2017 at 11:46 AM, Atri Sharma <[email protected]> wrote: > Hi Wes, > > Thanks for your response. > > My main use case is that I want to introduce updatability to Parquet > records without going the route of replacing the entire page. > > Is that something that has already been discussed please? > > Regards, > > Atri > > On Mon, Dec 4, 2017 at 10:10 PM, Wes McKinney <[email protected]> wrote: >> hi Atri, >> >> From a prior discussion on the mailing list, it is not clear that this >> is a problem that concerns either Parquet format or the >> implementations in the Apache Parquet project. If data must be edited >> or deleted, then the point-of-truth Parquet files must be scanned and >> overwritten with the offending records deleted. Modifying files in >> place is not feasible due to the compression and encoding schemes >> (dictionary, run-length encoding) used in the Parquet format. Let me >> know if I am misunderstanding the use case. >> >> Thanks >> Wes >> >> On Mon, Dec 4, 2017 at 11:30 AM, Atri Sharma <[email protected]> wrote: >>> Hi Folks, >>> >>> Any update? >>> >>> On Fri, Dec 1, 2017 at 9:23 AM, Atri Sharma <[email protected]> wrote: >>>> https://issues.apache.org/jira/browse/PARQUET-1155 >>>> >>>> Anybody working on it? Can I take it up? >>> >>> >>> >>> -- >>> Regards, >>> >>> Atri >>> l'apprenant > > > > -- > Regards, > > Atri > l'apprenant
