Hi,
I am running some experiments with Parquet and Carbondata. I observe
that Carbondata is not recording statistics for Spark to display in
Spark UI (history server). I run a projection queries both on Parquet
(sqlContext.sql()) and Carbondata (carbonContext.sql()). Parquet is
showing the complete details (total tasks and # of disk read in bytes)
whereas Carbondata is not showing anything. I would like to have these
statistics for carbondata. It helps to see how much Carbondata is
reading from the disk which is very useful while comparing it with other
formats.
I have also attached screenshot to make it easy to understand. I am
using latest Carbondata versionapache-carbondata-1.1.0.
<https://github.com/apache/carbondata/releases/tag/apache-carbondata-1.1.0>
Regards
Faisal
- Carbondata is not showing stats Rana Faisal
-