Kristian Waagan wrote:
The new stats are stored in the daemon. There is logic in
GenericStatement (at the very end of prepMinion) to go to the daemon to
fetch any new stats if they exists. Although all new stats will be
written at once, the decision on whether to go to the daemon to ask for
new stats is taken based on the statistics needed by the query being
compiled.
Another issue here is whether doing these checks only at compile time is
good enough. May the system reach a steady state where the stats are
becoming outdated and no queries are (re)compiled?
Ok, now I think I understand the async and how you are using the user
thread. I had not considered using subsequent executions to do the
system catalog update. I think it would be a little less complicated
if we could figure out a way to just let the daemon do the update, but
previous attempts did run into problems. I can definitely see cases
where we may never compile again in a server session, and it would be
sad to do all the work to create the stats and never get them written.
I think the most important case to handle is where no stats exist at
all. For a table of any significant size stats should not become
outdated, and determining if they are outdated may take just as much
work as compiling them again. For derby the only stats that we are
talking about here is the cardinality - basically how many duplicates
per key. static histogram stats are not kept, instead at compile time
the actual indexes are used for histogram based costing of key ranges.
maybe longer term we can invalidate based on large changes in number
of rows, or just date that last statistics were gathered. And really
longer term we could invalidate if we figure out during execution that
compile time estimate was wrong.
