lindong28 commented on a change in pull request #5: URL: https://github.com/apache/flink-ml/pull/5#discussion_r716167186
########## File path: flink-ml-iteration/src/main/java/org/apache/flink/ml/iteration/IterationBody.java ########## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.ml.iteration; + +/** The builder of the subgraph that will be executed inside the iteration. */ +public interface IterationBody { Review comment: Given that this is a public interface, do we need @PublicEvolving here? ########## File path: flink-ml-iteration/src/main/java/org/apache/flink/ml/iteration/DataStreamList.java ########## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.ml.iteration; + +import org.apache.flink.streaming.api.datastream.DataStream; + +import java.util.Arrays; +import java.util.Collections; +import java.util.List; + +/** An utility class to maintain a list of {@link DataStream}, which might have different types. */ +public class DataStreamList { + public static DataStreamList of(DataStream<?>... streams) { + return new DataStreamList(Arrays.asList(streams)); Review comment: Hmm... It seems that this static method is added to the FLIP-176 since our last discussion of the API section0. I am not sure we need this static method given that we already have `public DataStreamList(List<DataStream<?>> dataStreams)`. Could you help explain the motivation for this method? ########## File path: flink-ml-iteration/src/main/java/org/apache/flink/ml/iteration/IterationBodyResult.java ########## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.ml.iteration; + +import org.apache.flink.streaming.api.datastream.DataStream; + +import javax.annotation.Nullable; + +/** The result of an iteration, specifying the feedbacks and the outputs. */ +public class IterationBodyResult { + + /** + * A list of feedback variable streams. These streams will only be used during the iteration + * execution and will not be returned to the caller of the iteration body. It is assumed that + * the method which executes the iteration body will feed the records of the feedback variable + * streams back to the corresponding input variable streams. + */ + private final DataStreamList feedbackVariableStreams; + + /** + * A list of output streams. These streams will be returned to the caller of the methods that + * execute the iteration body. + */ + private final DataStreamList outputStreams; + + /** + * An optional termination criteria stream. If this stream is not null, it will be used together + * with the feedback variable streams to determine when the iteration should terminate. + */ + private final @Nullable DataStream<?> terminationCriteria; Review comment: The FLIP-176 uses `Optional<DataStream<?>>`. Can you explain what is the rational to choose `@Nullable DataStream<?>`? For example, is it more consistent with the existing pattern in Flink? And hopefully we can keep the PR consistent with the FLIP. ########## File path: flink-ml-iteration/src/main/java/org/apache/flink/ml/iteration/IterationBodyResult.java ########## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.ml.iteration; + +import org.apache.flink.streaming.api.datastream.DataStream; + +import javax.annotation.Nullable; + +/** The result of an iteration, specifying the feedbacks and the outputs. */ +public class IterationBodyResult { + + /** + * A list of feedback variable streams. These streams will only be used during the iteration + * execution and will not be returned to the caller of the iteration body. It is assumed that + * the method which executes the iteration body will feed the records of the feedback variable + * streams back to the corresponding input variable streams. + */ + private final DataStreamList feedbackVariableStreams; Review comment: Could you update the FLIP-176 to include e.g. `private final` to keep the API consistent with FLIP-176? ########## File path: flink-ml-iteration/src/main/java/org/apache/flink/ml/iteration/Iterations.java ########## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.ml.iteration; + +import org.apache.flink.annotation.PublicEvolving; + +/** A helper class to create iterations. */ +@PublicEvolving +public class Iterations { + Review comment: Hmm... it seems that the FLIP uses the class name `IterationUtils`. We will need to update the FLIP first if we agree to use this name. I would slightly prefer `IterationUtils` because this class provides just static public methods. But I am not strong on this. Can you explain the rational for `Iterations`? We can also ask other reviewers for opinions. ########## File path: flink-ml-iteration/src/main/java/org/apache/flink/ml/iteration/IterationBody.java ########## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.ml.iteration; + +/** The builder of the subgraph that will be executed inside the iteration. */ +public interface IterationBody { + + /** + * This method creates the graph for the iteration body. See {@link Iterations} for how the Review comment: It seems that the Java doc is different from FLIP-176. It will be nice if we can keep Java doc here consistent with the FLIP. Among other things, can we keep the requirement that "the number of feedback variable streams returned by this method must equal the number of variable streams given to this method." ########## File path: flink-ml-iteration/src/main/java/org/apache/flink/ml/iteration/IterationBodyResult.java ########## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.ml.iteration; + +import org.apache.flink.streaming.api.datastream.DataStream; + +import javax.annotation.Nullable; + +/** The result of an iteration, specifying the feedbacks and the outputs. */ +public class IterationBodyResult { + + /** + * A list of feedback variable streams. These streams will only be used during the iteration + * execution and will not be returned to the caller of the iteration body. It is assumed that + * the method which executes the iteration body will feed the records of the feedback variable + * streams back to the corresponding input variable streams. + */ + private final DataStreamList feedbackVariableStreams; + + /** + * A list of output streams. These streams will be returned to the caller of the methods that + * execute the iteration body. + */ + private final DataStreamList outputStreams; + + /** + * An optional termination criteria stream. If this stream is not null, it will be used together + * with the feedback variable streams to determine when the iteration should terminate. + */ + private final @Nullable DataStream<?> terminationCriteria; + + public IterationBodyResult( + DataStreamList feedbackVariableStreams, DataStreamList outputStreams) { + this(feedbackVariableStreams, outputStreams, null); + } + + public IterationBodyResult( + DataStreamList feedbackVariableStreams, + DataStreamList outputStreams, + @Nullable DataStream<?> terminationCriteria) { + this.feedbackVariableStreams = feedbackVariableStreams; + this.outputStreams = outputStreams; + this.terminationCriteria = terminationCriteria; + } + + public DataStreamList getFeedbackVariableStreams() { Review comment: Since all members are `final`, can we just make them `public final` and remove the accessor methods here? ########## File path: flink-ml-iteration/pom.xml ########## @@ -0,0 +1,42 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> +<project xmlns="http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.apache.flink</groupId> + <artifactId>flink-ml-parent</artifactId> + <version>0.1-SNAPSHOT</version> + </parent> + + <artifactId>flink-ml-iteration</artifactId> Review comment: Is there any case that user will want to import just one of the flink-ml-iteration module and the flink-ml-api module, but not the other? Would it be simpler to merge the flink-ml-iteration module with the flink-ml-api module? We can rename the flink-ml-api module as appropriate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
