[ https://issues.apache.org/jira/browse/FLINK-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635206#comment-14635206 ]
Till Rohrmann commented on FLINK-1901: -------------------------------------- I think this solution is indeed a little bit too "hacky". It would be very unintuitive for the user having to broadcast the iteration {{DataSet}} to the sampling operator. Furthermore, this will inflict unnecessary network I/O. I think we should try to solve this problem properly. This means that we have a single sampling operator which works inside and outside of iterations. This will also avoid code duplication. > Create sample operator for Dataset > ---------------------------------- > > Key: FLINK-1901 > URL: https://issues.apache.org/jira/browse/FLINK-1901 > Project: Flink > Issue Type: Improvement > Components: Core > Reporter: Theodore Vasiloudis > Assignee: Chengxiang Li > > In order to be able to implement Stochastic Gradient Descent and a number of > other machine learning algorithms we need to have a way to take a random > sample from a Dataset. > We need to be able to sample with or without replacement from the Dataset, > choose the relative size of the sample, and set a seed for reproducibility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)