[
https://issues.apache.org/jira/browse/FLINK-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15044876#comment-15044876
]
ASF GitHub Bot commented on FLINK-2379:
---------------------------------------
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/1032#issuecomment-162518625
Hi @sachingoel0101, I just reviewed quickly. I have some questions and
comments for this pull request.
First, why `FieldStats` covers statistics value for continuous and discrete
values? There is no common value between statistics value for continuous and
that for discrete. How about split them into `ContinuousFieldStats` and
`DiscreteFieldStats`?
Second, many scaladocs are written in javadoc style.
Third, we need to generalize type of the input dataset to `DataSet[T]`
whose generic parameter is `T <: Vector` because `DataSet[Vector]` cannot cover
`DataSet[DenseVector]` and `DataSet[SparseVector]`.
If there is something which I misunderstood, please feel free to leave a
comment.
> Add methods to evaluate field wise statistics over DataSet of vectors.
> ----------------------------------------------------------------------
>
> Key: FLINK-2379
> URL: https://issues.apache.org/jira/browse/FLINK-2379
> Project: Flink
> Issue Type: New Feature
> Components: Machine Learning Library
> Reporter: Sachin Goel
> Assignee: Sachin Goel
>
> Design methods to evaluate statistics over dataset of vectors.
> For continuous fields, Minimum, maximum, mean, variance.
> For discrete fields, Class counts, Entropy, Gini Impurity.
> Further statistical measures can also be supported. For example, correlation
> between two series, computing the covariance matrix, etc.
> [These are currently the things Spark supports.]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
