Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/14712
Ok let's focus on this PR. You use the same table property names of Hive
for the new statistics, then what if they are changed by Hive unexpectedly? Are
they supported by all versions of Hive?
I think it's good to not mix the target 1 and 2 mentioned in my last
comment together, and what do you think about my proposal in
https://github.com/apache/spark/pull/14712#issuecomment-241207069? We can hide
the hive-compatible logic in `HiveExternalCatalog`, so that `InMemoryCatalog`
can support it too.
BTW in the view of API, `InMemoryCatalog` is also a persistent level, table
statistics should also work in `InMemoryCatalog`.
> Of course we can put everything into properties, but it's not elegant.
But sometimes we have to, like data source table metadata. I think it's ok
if we can hide the logic in `HiveExternalCatalog`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
