AngersZhuuuu opened a new pull request, #36377:
URL: https://github.com/apache/spark/pull/36377
### What changes were proposed in this pull request?
When use `InsertIntoHiveTable`, when insert overwrite partition, it will call
`Hive.loadPartition()`, in this method, when `hive.stats.autogather` is
true(default is true)
```
if (oldPart == null) {
newTPart.getTPartition().setParameters(new HashMap<String,String>());
if
(this.getConf().getBoolVar(HiveConf.ConfVars.HIVESTATSAUTOGATHER)) {
StatsSetupConst.setBasicStatsStateForCreateTable(newTPart.getParameters(),
StatsSetupConst.TRUE);
}
public static void setBasicStatsStateForCreateTable(Map<String, String>
params, String setting) {
if (TRUE.equals(setting)) {
for (String stat : StatsSetupConst.supportedStats) {
params.put(stat, "0");
}
}
setBasicStatsState(params, setting);
}
public static final String[] supportedStats =
{NUM_FILES,ROW_COUNT,TOTAL_SIZE,RAW_DATA_SIZE};
```
it set default `numRow` and `numFiles` as 0, since spark will update
numFiles and rawSize, so `numRow` remain 0.
This impact other system like presto's CBO. Since spark have it's own
statistic related code, here should set this as false by default.
### Why are the changes needed?
Keep consist with spark's own statistic data
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
