[GitHub] dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573: Consistent API for Ensemble training
dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573:
Consistent API for Ensemble training
URL: https://github.com/apache/ignite/pull/5767#discussion_r247937061
##
File path:
modules/ml/src/main/java/org/apache/ignite/ml/composition/combinators/parallel/TrainersParallelComposition.java
##
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.ml.composition.combinators.parallel;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import org.apache.ignite.ml.IgniteModel;
+import org.apache.ignite.ml.composition.CompositionUtils;
+import org.apache.ignite.ml.dataset.DatasetBuilder;
+import org.apache.ignite.ml.environment.parallelism.Promise;
+import org.apache.ignite.ml.math.functions.IgniteBiFunction;
+import org.apache.ignite.ml.math.functions.IgniteSupplier;
+import org.apache.ignite.ml.math.primitives.vector.Vector;
+import org.apache.ignite.ml.trainers.DatasetTrainer;
+
+/**
+ * This class represents a parallel composition of trainers.
+ * Parallel composition of trainers is a trainer itself which trains a list of
trainers with same
+ * input and output. Training is done in following manner:
+ *
+ * 1. Independently train all trainers on the same dataset and get a list
of models.
+ * 2. Combine models produced in step (1) into a {@link
ModelsParallelComposition}.
+ *
+ * Updating is made in a similar fashion.
+ * Like in other trainers combinators we avoid to include type of contained
trainers in type parameters
+ * because otherwise compositions of compositions would have a relatively
complex generic type which will
+ * reduce readability.
+ *
+ * @param Type of trainers inputs.
+ * @param Type of trainers outputs.
+ * @param Type of dataset labels.
+ */
+public class TrainersParallelComposition extends
DatasetTrainer>, L> {
+/** List of trainers. */
+private final List, L>> trainers;
+
+/**
+ * Construct an instance of this class from a list of trainers.
+ *
+ * @param trainers Trainers.
+ * @param Type of model.
+ * @param Type of trainer.
+ */
+public , T extends DatasetTrainer, L>> TrainersParallelComposition(
Review comment:
We don't need `M` type.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573: Consistent API for Ensemble training
dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573:
Consistent API for Ensemble training
URL: https://github.com/apache/ignite/pull/5767#discussion_r247541135
##
File path:
modules/ml/src/main/java/org/apache/ignite/ml/dataset/UpstreamTransformer.java
##
@@ -22,29 +22,15 @@
/**
* Interface of transformer of upstream.
- *
- * @param Type of keys in the upstream.
- * @param Type of values in the upstream.
*/
// TODO: IGNITE-10297: Investigate possibility of API change.
@FunctionalInterface
-public interface UpstreamTransformer extends Serializable {
+public interface UpstreamTransformer extends Serializable {
Review comment:
Why do we remove types here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573: Consistent API for Ensemble training
dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573:
Consistent API for Ensemble training
URL: https://github.com/apache/ignite/pull/5767#discussion_r247542728
##
File path:
modules/ml/src/main/java/org/apache/ignite/ml/trainers/AdaptableDatasetTrainer.java
##
@@ -56,27 +68,46 @@
* @param Type of labels.
* @return Instance of this class.
*/
-public static , L>
AdaptableDatasetTrainer of(DatasetTrainer wrapped) {
-return new AdaptableDatasetTrainer<>(IgniteFunction.identity(),
wrapped, IgniteFunction.identity());
+public static , L>
AdaptableDatasetTrainer of(
Review comment:
Oh, so many types...
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573: Consistent API for Ensemble training
dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573:
Consistent API for Ensemble training
URL: https://github.com/apache/ignite/pull/5767#discussion_r247492424
##
File path:
modules/ml/src/main/java/org/apache/ignite/ml/composition/combinators/parallel/ModelsParallelComposition.java
##
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.ml.composition.combinators.parallel;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import org.apache.ignite.ml.IgniteModel;
+
+/**
+ * Parallel composition of models.
+ * Parallel composition of models is a model which contains a list of
submodels with same input and output types.
+ * Result of prediction in such model is a list of predictions of each of
submodels.
+ *
+ * @param Type of submodel input.
+ * @param Type of submodel output.
+ */
+public class ModelsParallelComposition implements IgniteModel> {
+/** List of submodels. */
+private final List> submodels;
+
+/**
+ * Construc an instance of this class from list of submodels.
+ *
+ * @param submodels List of submodels constituting this model.
+ */
+public ModelsParallelComposition(List> submodels) {
+this.submodels = submodels;
+}
+
+/** {@inheritDoc} */
+@Override public List predict(I i) {
+return submodels
+.stream()
+.map(m -> m.predict(i))
+.collect(Collectors.toList());
+}
+
+/**
+ * List of submodels constituting this model.
+ *
+ * @return List of submodels constituting this model.
+ */
+public List> submodels() {
+return new ArrayList<>(submodels);
Review comment:
Would `Collections.unmodifiableList(submodels)` be better in this case?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573: Consistent API for Ensemble training
dmitrievanthony commented on a change in pull request #5767: [ML] IGNITE-10573:
Consistent API for Ensemble training
URL: https://github.com/apache/ignite/pull/5767#discussion_r247540682
##
File path:
modules/ml/src/main/java/org/apache/ignite/ml/composition/stacking/StackedDatasetTrainer.java
##
@@ -402,11 +346,12 @@ public StackedDatasetTrainer() {
IgniteBiFunction featureExtractor,
IgniteBiFunction lbExtractor) {
// This method is never called, we override "update" instead.
-return null;
+throw new IllegalStateException();
}
/** {@inheritDoc} */
@Override protected boolean checkState(StackedModel mdl) {
-return true;
+// Should be never called.
+throw new IllegalStateException();
Review comment:
Please, update javadoc.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
