GitHub user gatorsmile reopened a pull request:
https://github.com/apache/spark/pull/14971
[SPARK-17410] [SPARK-17284] Move Hive-generated Stats Info to HiveClientImpl
### What changes were proposed in this pull request?
After we adding a new field `stats` into `CatalogTable`, we should not
expose Hive-specific Stats metadata to `MetastoreRelation`. It complicates all
the related codes. It also introduces a bug in `SHOW CREATE TABLE`. The
statistics-related table properties should be skipped by `SHOW CREATE TABLE`,
since it could be incorrect in the newly created table. See the Hive JIRA:
https://issues.apache.org/jira/browse/HIVE-13792
This PR is to handle Hive-specific Stats metadata in `HiveClientImpl`.
### How was this patch tested?
Added a few test cases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark showCreateTableNew
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14971.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14971
----
commit 92474c5a142fb9db2c86549c8347f910fc01fcbd
Author: gatorsmile <[email protected]>
Date: 2016-08-28T22:28:15Z
remove stats-related props
commit ce8e8b89a5b61648daaa59578e2b6a99ec2f6d74
Author: gatorsmile <[email protected]>
Date: 2016-08-29T05:24:05Z
address comments
commit 9ce526b7729c4111292d6abb69bd81aec0ecf5de
Author: gatorsmile <[email protected]>
Date: 2016-09-06T00:25:56Z
Merge remote-tracking branch 'upstream/master' into showCreateTable
commit efd879dbda12e235d00d9b6bc4891a591832912b
Author: gatorsmile <[email protected]>
Date: 2016-09-06T07:07:12Z
move stats from metastorerelation to hiveclientimpl
commit 491c0cd2580cfd358b2fd4d94a6f41b4063f172f
Author: gatorsmile <[email protected]>
Date: 2016-09-06T07:34:51Z
improve the comments.
commit c9cdf44b561c5e88a108cd09ad17842332d14162
Author: gatorsmile <[email protected]>
Date: 2016-09-06T07:35:26Z
improve the comments.
commit 4b0aed54b0aef6675f4f9fac82f6647563afb5cc
Author: gatorsmile <[email protected]>
Date: 2016-09-08T03:21:28Z
Merge remote-tracking branch 'upstream/master' into showCreateTable
commit 552101af418e4a30febf7d09938022ecc4c08da9
Author: gatorsmile <[email protected]>
Date: 2016-09-08T03:25:00Z
merge
commit d3dcb564509fd2a32a3fadefb811495affaaa466
Author: gatorsmile <[email protected]>
Date: 2016-09-11T05:55:20Z
Merge remote-tracking branch 'upstream/master' into showCreateTable
commit 9e18ba104527d2bb14331f4b51194002dabb2556
Author: gatorsmile <[email protected]>
Date: 2016-09-11T21:48:36Z
fix and add more test cases
commit c6a85bcd4b6b58c46787d1ca1510418cef79a8d5
Author: gatorsmile <[email protected]>
Date: 2016-09-16T04:50:18Z
Merge remote-tracking branch 'upstream/master' into showCreateTable
commit 3ed68e0b0aa6aff19a5e31c89fed7e5c814e83f3
Author: gatorsmile <[email protected]>
Date: 2016-09-16T06:16:40Z
improve the test case
commit 2e4d398388cd64f3e1d130af81d5e7ddc23a2a19
Author: gatorsmile <[email protected]>
Date: 2016-09-17T06:52:32Z
also utilizes Hive-generated row counts when not analyzed in Spark
commit 5dfa17efa84ed180e68b4922cfaf85e3d50f14ad
Author: gatorsmile <[email protected]>
Date: 2016-09-17T07:30:57Z
more comments
commit 2f40c7f5532c8b6e66c786f3b1506bd4efdcf711
Author: gatorsmile <[email protected]>
Date: 2016-09-18T00:08:48Z
address comments.
commit 3376bd6a57a65fa004abd43237f8f3c87f07064a
Author: gatorsmile <[email protected]>
Date: 2016-09-18T03:31:11Z
fix test cases
commit 90cd18e9d7bad6462fb0254d7981e23341795c11
Author: gatorsmile <[email protected]>
Date: 2016-09-21T04:58:42Z
Merge remote-tracking branch 'upstream/master' into showCreateTable
commit 7ad08fe2a488fa759b4abf4e99a7206e031379d9
Author: gatorsmile <[email protected]>
Date: 2016-09-21T05:23:58Z
test case fix
commit f4c0ebb0901216ea09eaf3f77e4fdcd431b15d37
Author: gatorsmile <[email protected]>
Date: 2016-09-22T23:08:55Z
address comments
commit 4c89d92ab65d7f4f061e32aa22780fd6e4b7c798
Author: gatorsmile <[email protected]>
Date: 2016-09-22T23:12:57Z
address comments
commit 699b5d8aa4d9370009c73f45d1618f1e5bb92210
Author: gatorsmile <[email protected]>
Date: 2016-09-24T01:10:04Z
fix.
commit 8c90d0b7364c46de5a4a59fa89457000bb283dd9
Author: gatorsmile <[email protected]>
Date: 2016-09-25T05:33:44Z
Merge remote-tracking branch 'upstream/master' into showCreateTable
commit 50ce04e51aebd5f68f7e50ec7c3bbe72275bf629
Author: gatorsmile <[email protected]>
Date: 2016-09-25T07:10:30Z
address comments.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
