[
https://issues.apache.org/jira/browse/DERBY-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244421#comment-13244421
]
Mike Matrigali commented on DERBY-5680:
---------------------------------------
I reviewed the change, could use a few more comments for those of us not
familar with data design of the statistics catalog, but I would
have expected it fix the problem. Not sure what the issue with the assert is,
there is a comment about when we turn on write mode, but
not sure when it is expected to get turned off - maybe at commit?
I was hoping the change could be less invasive by just dropping all statistics
even bad ones upfront with no checking, ie. not have to reget all the valid
rows and recheck them always. But I see the current
code logic wants to drop statistics one at a time, and that were problems
trying to invalidate all at once which might be similar to problems with
dropping all rows at once.
It is not part of this change, but is there a possible problem with when
invalidateStatements is called? Unless you need it for some
locking reason it would seem better to invalidate after you do the update than
in the middle.
Can queries sneak in between when a statistic is dropped and when you insert
the new updated statistic? If so it would be another reason
not to drop them all up front as it would increase the window. brett was
reporting some wierdness with working queries getting recompiled
to bad plans. I think we can think of his system as always having concurrent
processors availble to run statements constantly, so they
can almost instantaneously start recompiling once you invalidate their plan.
Now if they can get in after the invalidate, after the drop but before
the insert then they probably have worse stats (ie. no stats), vs the old stats.
> indexStat daemon processing tables over an over even when there are no
> changes in the tables
> --------------------------------------------------------------------------------------------
>
> Key: DERBY-5680
> URL: https://issues.apache.org/jira/browse/DERBY-5680
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.8.2.2
> Reporter: Brett Bergquist
> Attachments: derby-5680-1a-drop_orphaned_stats.diff
>
>
> I think there is something wrong with the indexStats.
> The problem happens on many tables in the database.
> None of these tables are changing however, no inserts or deletes or updates.
> They are being queried, however.
> Here is one such table.
> Here is the statistics for this table:
> Table (Index) 2 3
> ACCOUNTTABLE_CONFIG_BUNDLE (SQL081029110443810) numunique= 38390
> numrows= 38390 2012-03-30 13:00:26.84
> ACCOUNTTABLE_CONFIG_BUNDLE (SQL100922215819290) numunique= 38390
> numrows= 38390 2012-03-30 13:00:26.917
> There are in fact 38390 rows in the table.
> Here is some of the indexStat trace:
> Fri Mar 30 12:47:12 EDT 2012 Thread[DRDAConnThread_43,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": update scheduled,
> reason=[t-est=38390, i-est=2355 => cmp=2.7912562815443245] (queueSize=12)
> Fri Mar 30 12:47:48 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL081029110443810 (fc33890d-011d-491f-3d8c-0000376d74d3): rows=38390,
> card=[38390]
> Fri Mar 30 12:47:48 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL100922215819290 (75608675-012b-3c38-b55c-000043ea6398): rows=38390,
> card=[38390]
> Fri Mar 30 12:47:48 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": scan durations
> (c30625=91ms,c30625=98ms)
> Fri Mar 30 12:47:48 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": generation complete (210 ms)
> Fri Mar 30 12:47:49 EDT 2012 Thread[DRDAConnThread_44,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": update scheduled,
> reason=[t-est=38390, i-est=2355 => cmp=2.7912562815443245] (queueSize=19)
> Fri Mar 30 12:48:25 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL081029110443810 (fc33890d-011d-491f-3d8c-0000376d74d3): rows=38390,
> card=[38390]
> Fri Mar 30 12:48:25 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL100922215819290 (75608675-012b-3c38-b55c-000043ea6398): rows=38390,
> card=[38390]
> Fri Mar 30 12:48:25 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": scan durations
> (c30625=93ms,c30625=95ms)
> Fri Mar 30 12:48:25 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": generation complete (211 ms)
> Fri Mar 30 12:48:25 EDT 2012 Thread[DRDAConnThread_50,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": update scheduled,
> reason=[t-est=38390, i-est=2355 => cmp=2.7912562815443245] (queueSize=18)
> Fri Mar 30 12:48:57 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL081029110443810 (fc33890d-011d-491f-3d8c-0000376d74d3): rows=38390,
> card=[38390]
> Fri Mar 30 12:48:57 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL100922215819290 (75608675-012b-3c38-b55c-000043ea6398): rows=38390,
> card=[38390]
> Fri Mar 30 12:48:57 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": generation complete (243 ms)
> Fri Mar 30 12:49:27 EDT 2012 Thread[DRDAConnThread_56,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": update scheduled,
> reason=[t-est=38390, i-est=2355 => cmp=2.7912562815443245] (queueSize=20)
> Fri Mar 30 12:49:36 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL081029110443810 (fc33890d-011d-491f-3d8c-0000376d74d3): rows=38390,
> card=[38390]
> Fri Mar 30 12:49:37 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": wrote stats for index
> SQL100922215819290 (75608675-012b-3c38-b55c-000043ea6398): rows=38390,
> card=[38390]
> Fri Mar 30 12:49:37 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": scan durations
> (c30625=111ms,c30625=108ms)
> Fri Mar 30 12:49:37 EDT 2012 Thread[index-stat-thread,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": generation complete (238 ms)
> Fri Mar 30 12:49:37 EDT 2012 Thread[DRDAConnThread_49,5,main] {istat}
> "PKG_9145E_V1"."ACCOUNTTABLE_CONFIG_BUNDLE": update scheduled,
> reason=[t-est=38390, i-est=2355 => cmp=2.7912562815443245] (queueSize=18)
> As can be seen, the "i-est" appears to be wrong and is used over and over
> even though the statistics for the indexes have been updated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
