Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/14712
@cloud-fan what i was saying is out of the range of this pr.
Table property is a string-string map, which means we need to transform
every statistic into/from a string. This is ok for simple table statistics like
"numRows", but not a good choice for complicated column statistics like
histograms.
InMemoryCatalog is "in memory", we can have our own properties or data
structures, but currently we still need Hive metastore api to âpersistâ
these statistics. Hive has APIs for storing/loading table properties and
ColumnStats, but no api for histograms.
What I'm trying to say is, we can use hive as a persistent level, but what
we can store/load is still limited by its api. Of course we can put everything
into properties, but it's not elegant.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
