Github user thvasilo commented on a diff in the pull request:
https://github.com/apache/flink/pull/861#discussion_r33146188
--- Diff:
flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/math/CategoricalHistogram.scala
---
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.ml.math
+
+import java.util
+
+/** Implementation of a discrete valued online histogram
+ *
+ */
+class CategoricalHistogram(
+ values: Array[Double],
+ counts: Array[Int] = Array.ofDim[Int](0))
+ extends OnlineHistogram
+ with Serializable {
+
+ // sanity checks
+ require(values.length > 0, "Number of classes should be a positive
integer")
+ require(counts.length == 0 || counts.length == values.length, "Counts
should have the same " +
+ "number" + " " + "of " + "entries as values")
+ val data = new util.ArrayList[(Double, Int)](values.length)
+ require(checkSanity, "All values in counts should be non-negative")
+
+ /** Number of bins currently being used
+ *
+ * @return number of bins
+ */
+ override def bins: Int = {
+ data.size
+ }
+
+ /** Adds a new instance
+ *
+ * @param p value to be added
+ */
+ override def add(p: Double): Unit = {
+ // search for the index where the value is equal to1 p
+ val search = find(p)
+ require(search >= 0, p + "is not present in the histogram")
+ val currentVal = data.get(search)
+ data.set(search, (currentVal._1, currentVal._2 + 1))
+ }
+
+ /** Merges the histogram with h and returns a new histogram
+ *
+ * @param h histogram to be merged
+ * @param B capacity of the resultant histogram
+ * @return Merged histogram with capacity B
+ */
+ override def merge(h: OnlineHistogram, B: Int): CategoricalHistogram = {
+ h match{
+ case h1: CategoricalHistogram => {
+ require(B == bins && B == h.bins, "Size of the histograms and B
should be equal")
--- End diff --
The semantics of the merge function change between the continuous and the
categorical case.
In the continuous case we are able to merge histograms with different
number of buckets, but that is not possible
in the discrete case.
Why not make it possible to merge discrete histograms as well? For h3 =
h1.merge(h2)
The buckets of the result h3 would be equal to the union of the buckets in
h1 and h2. Am I missing something here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
