[
https://issues.apache.org/jira/browse/PHOENIX-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466376#comment-16466376
]
James Taylor commented on PHOENIX-4724:
---------------------------------------
Great stuff, [~vincentpoon]. Couple of questions:
* Does the histogram building take into account when a value is being
overwritten again and again (since at write time we don't know if we're
overwriting or not, unless the table is declared immutable)? Same question on
delete of a column value.
* How would you envision this be integrated with Phoenix? Maybe we could
generate it when we update stats: at table compaction time and when UPDATE
STATISTICS is called? Or depending on answers to above, would we try to
constantly update the histogram as the data is mutating?
* This information would definitely be useful for the optimizer. We have
PHOENIX-1178, but not much detail there. [~maryannxue] could likely fill in how
this information could be used. One big area is in estimating how much data
will be filtered with a WHERE clause. Another is in how distinct a column is.
* Did you write EquiDepthStreamHistogram.java or did you find it somewhere
online? If that latter, how was it licensed?
> Efficient Equi-Depth histogram for streaming data
> -------------------------------------------------
>
> Key: PHOENIX-4724
> URL: https://issues.apache.org/jira/browse/PHOENIX-4724
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: Vincent Poon
> Assignee: Vincent Poon
> Priority: Major
> Attachments: PHOENIX-4724.v1.patch
>
>
> Equi-Depth histogram from
> http://web.cs.ucla.edu/~zaniolo/papers/Histogram-EDBT2011-CamReady.pdf, but
> without the sliding window - we assume a single window over the entire data
> set.
> Used to generate the bucket boundaries of a histogram where each bucket has
> the same # of items.
> This is useful, for example, for pre-splitting an index table, by feeding in
> data from the indexed column.
> Works on streaming data - the histogram is dynamically updated for each new
> value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
