[
https://issues.apache.org/jira/browse/TRAFODION-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15589235#comment-15589235
]
ASF GitHub Bot commented on TRAFODION-2282:
-------------------------------------------
GitHub user DaveBirdsall opened a pull request:
https://github.com/apache/incubator-trafodion/pull/766
[TRAFODION-2282][TRAFODION-882] New scheme to invalidate histogram cache
This set of code changes leverages the query invalidation and security key
invalidation infrastructure to invalidate the histogram cache. The design is as
follows:
1. At UPDATE STATISTICS time, after statistics have been written, a new
stats invalidation key is generated. The key contains the table UID of the
table whose stats have been updated.
2. RMS propagates this key to all compilers in the cluster. Note that the
master executors need not know, as changing histograms does not invalidate
query plans. (Hence SscpNewIncomingConnectionStream::processSecInvReq ignores
stats invalidation keys.)
3. The compiler main, when checking for invalidation keys it has not seen
before, processes the new stats invalidation keys. It removes from the
HistogramCache any HistogramCacheEntry that has the same table UID.
4. Fail safe logic to invalidate the entire histogram cache was added in
the case that the compiler has been idle so long that it may have missed seeing
a stats invalidation key.
Older code in HistogramCache::getHistograms (optimizer/NATable.cpp) that
used a refresh interval timing scheme to invalidate cache entries has been
completely removed. As part of this cleanup, two CQDs are no longer needed and
are removed. Stubbed routines in ustats/hs_read.cpp have been removed. Too, it
was noticed that the modified time stamp was stubbed (and cannot be
un-stubbed), and so that was removed. Note that the redefinition time stamp is
also stubbed but it is so pervasive in so many data structures I chose to leave
it alone for now.
Several regression test expected files have been updated, as UPDATE
STATISTICS now takes effect immediately in a session resulting in plan changes
and related data.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DaveBirdsall/incubator-trafodion Trafodion2282
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/766.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #766
----
commit 9a3cf2e53af6d9df7c8ec03665f5da5a0ebf8ab0
Author: Dave Birdsall <[email protected]>
Date: 2016-10-19T16:49:51Z
[TRAFODION-2282][TRAFODION-882] New scheme to invalidate histogram cache
----
> Optimizer histogram cache is sometimes never refreshed
> ------------------------------------------------------
>
> Key: TRAFODION-2282
> URL: https://issues.apache.org/jira/browse/TRAFODION-2282
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.1-incubating
> Environment: All
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
> Fix For: 2.1-incubating
>
>
> Here's the scenario:
> In session 1, create a table T1 and populate it with, say, 200000 or more
> rows.
> In session 2, prepare a query that selects from T1. For example, select
> A,COUNT(*) FROM T1 GROUP BY A. The query should compile fine, but you should
> see missing statistics warning messages (warning codes 6007 and 6008).
> In session 1, do "update statistics for table t1 on every column"
> In session 2, prepare other queries over a period of time. What should happen
> is that after about an hour, the Optimizer decides to refresh the histograms.
> However, due to a bug, it might never do so. So, the missing statistics
> warnings (6007, 6008) never go away. Worse, the prepares never benefit from
> the statistics so plan choice may be poor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
