Hey all - I posted this to both dev and user as I could mentally make the argument for both,
Sorry if this is answered somewhere already. I know in the past, there have been discussions around using two different readers for Parquet, and performance gains/losses, issues. etc. Right now, the store.parquet.use_new_reader is set to false (1.10) and I was trying to get more information about this and what the eventual roadmap for drill will be... So, in the docs I see use_new_reader is set to be not supported in this release. What I am looking for is a little information on: - What the two readers are (is one a special drill thing, is the other a standard reader from the parquet project?) - What is the eventual goal here... to be able to use and switch between both? To provide the option? To have code parity with another project? - Do either of the readers work with Arrow? - How does Arrow and Parquet readers fit together? - Will the readers ever converge? I have other questions too, but these are examples of where I am coming from. Is there a good starting place for my research on this subject? Thanks, John
