Re: Small malloc at file open and metadata parsing

Sorry to jump in like this, but I was wondering if parquet-rs can read the file correctly, or the issue also happens there. Alex, could you give it a go and see if file and metadata can be read with parquet-rs (https://github.com/sunchao/parquet-rs, you can run cargo install parquet to install

Re: Small malloc at file open and metadata parsing

Thanks for the quick reply @Wes, Too bad this is causing a lot of delays (due to page fault handing) for light queries (ones that query only few rows/columns), Will try to use jemallc and see,,, One more question, when i upgrade to 1.4.0 or later code, and use the same cmake options, and

Re: Small malloc at file open and metadata parsing

hi Alex, It looks like the mallocs are coming from Thrift (parquet/parquet_types.cpp is generated by Thrift). I'm not sure if we can do much about this. I'm curious if it's possible to pass a custom STL allocator to Thrift so we could use a different allocation strategy than the default STL

Small malloc at file open and metadata parsing

Hi, I'm reading parquet file (generated by Java parquet library). Our schema has 400 columns (including non-array elements, 1-dimensional array elements). I'm using release 1.3.1, gcc 4.8.5, boost static library 1.53, I compile parquet-cpp with following cmake options, ``` cmake3