leerho commented on issue #700: URL: https://github.com/apache/datasketches-java/issues/700#issuecomment-3647962132
@thomasrebele, You should read [this tutorial](https://datasketches.apache.org/docs/QuantilesAll/SketchingQuantilesAndRanksTutorial.html). In there you will find a set of "rules", which are effectively invariants of the sketch. In short, a sketch will only return a quantile or its associated rank that actually exists in the input stream. If you are interpolating, you are inventing data that may not exist in the stream. This will violate the invariants of the sketch and corrupt the ability of the sketch to capture the actual distribution of the input stream. If you need higher resolution of your stream, use a larger sketch (bigger k). Or if you are interested in higher resolution at one end of the rank spectrum you might switch to the REQ sketch. The confidence interval with a K=200 of the KLL sketch is about 1.65%. This means the resolving power of the sketch at that size is .0165 of the normalized rank range. Since the ranks are in the range (0,1], that means the sketch is unlikely to be able to resolve quantiles whose ranks are closer than .0165. And as Alex pointed out you are trying to resolve quantiles whose ranks are only .00001 apart. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
