HippoBaro commented on PR #23851:
URL: https://github.com/apache/datafusion/pull/23851#issuecomment-7018543672

   Thanks @alamb! Small clarification: my current work is on the writer side: 
preserving Arrow’s encoding characteristics ()including REE) through to Parquet 
without expanding intermediate representations. There are similar opportunities 
on the reader side that I’d like to explore next.
   
   I agree with the broad direction: Arrow is designed for in-memory processing 
and Parquet for storage. These objectives often compete with each other. I’d 
rather focus on making that transition efficient and encoding-preserving than 
build general execution over Parquet encodings directly, though retaining 
opportunities for early pruning is important.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to