Zoltan Haindrich created HIVE-23776:

             Summary: Retire quickstats autocollection
                 Key: HIVE-23776
                 URL: https://issues.apache.org/jira/browse/HIVE-23776
             Project: Hive
          Issue Type: Improvement
            Reporter: Zoltan Haindrich
            Assignee: Zoltan Haindrich

this is about:
* num files
* datasize (sum of filesizes)
* num erasure coded files

right now these are scanned during every BasicStatsTask execution - which means 
some filesystem reads/etc - for small inserts these are visible in case the fs 
is a bit slower (s3 and friends)

I don't think they are really in use...we rely more on columnstats which are 
more accurate ; and because of the datasize in this case is for "offline" 
(ondisk) - while we should be insted calculate with "online" sizes...


* remove collection and storage of this data
* collect it on the fly during "desc formatted" statements to provide them for 
informational purposes

This message was sent by Atlassian Jira

Reply via email to