[GitHub] [flink] flinkbot edited a comment on issue #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy…
flinkbot edited a comment on issue #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy… URL: https://github.com/apache/flink/pull/8912#issuecomment-527387437 ## CI report: * 1bd21c6cd2d430b9827b903e80f31c55bcb04750 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125569862) * 1b9269f48bd6b55ddd0f21faabbd648c706003a2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127308001) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy…
flinkbot edited a comment on issue #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy… URL: https://github.com/apache/flink/pull/8912#issuecomment-527387437 ## CI report: * 1bd21c6cd2d430b9827b903e80f31c55bcb04750 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125569862) * 1b9269f48bd6b55ddd0f21faabbd648c706003a2 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/127308001) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy…
flinkbot edited a comment on issue #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy… URL: https://github.com/apache/flink/pull/8912#issuecomment-527387437 ## CI report: * 1bd21c6cd2d430b9827b903e80f31c55bcb04750 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125569862) * 1b9269f48bd6b55ddd0f21faabbd648c706003a2 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhuzhurk commented on a change in pull request #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy…
zhuzhurk commented on a change in pull request #8912: [FLINK-12709] [runtime]
Implement new generation restart strategy loader which also respects legacy…
URL: https://github.com/apache/flink/pull/8912#discussion_r324026898
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/RestartBackoffTimeStrategyFactoryLoader.java
##
@@ -0,0 +1,393 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.executiongraph.failover.flip1;
+
+import org.apache.flink.api.common.restartstrategy.RestartStrategies;
+import org.apache.flink.configuration.AkkaOptions;
+import org.apache.flink.configuration.ConfigConstants;
+import org.apache.flink.configuration.ConfigOption;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.configuration.RestartBackoffTimeStrategyOptions;
+import
org.apache.flink.runtime.executiongraph.restart.FixedDelayRestartStrategy;
+import org.apache.flink.runtime.executiongraph.restart.NoRestartStrategy;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.concurrent.TimeUnit;
+
+import scala.concurrent.duration.Duration;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * A utility class to load {@link RestartBackoffTimeStrategy.Factory} from the
configuration.
+ */
+public class RestartBackoffTimeStrategyFactoryLoader {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(RestartBackoffTimeStrategyFactoryLoader.class);
+
+ private static final String CREATE_METHOD = "createFactory";
+
+ /**
+* Creates proper {@link RestartBackoffTimeStrategy.Factory}.
+* If new version restart strategy is specified, will directly use it.
+* Otherwise will decide based on legacy restart strategy configs.
+*
+* @param jobRestartStrategyConfiguration restart configuration given
within the job graph
+* @param clusterConfiguration cluster(server-side) configuration,
usually represented as jobmanager config
+* @param isCheckpointingEnabled if checkpointing was enabled for the
job
+* @return new version restart strategy factory
+*/
+ public static RestartBackoffTimeStrategy.Factory
createRestartStrategyFactory(
+ final RestartStrategies.RestartStrategyConfiguration
jobRestartStrategyConfiguration,
+ final Configuration clusterConfiguration,
+ final boolean isCheckpointingEnabled) throws Exception {
+
+ checkNotNull(jobRestartStrategyConfiguration);
+ checkNotNull(clusterConfiguration);
+
+ final String restartStrategyClassName =
clusterConfiguration.getString(
+
RestartBackoffTimeStrategyOptions.RESTART_BACKOFF_TIME_STRATEGY_CLASS_NAME);
+
+ if (restartStrategyClassName != null) {
+ // create new restart strategy directly if it is
specified in cluster config
+ return
createRestartStrategyFactoryInternal(clusterConfiguration);
+ } else {
+ // adapt the legacy restart strategy configs as new
restart strategy configs
+ final Configuration adaptedConfiguration =
getAdaptedConfiguration(
+ jobRestartStrategyConfiguration,
+ clusterConfiguration,
+ isCheckpointingEnabled);
+
+ // create new restart strategy from the adapted config
+ return
createRestartStrategyFactoryInternal(adaptedConfiguration);
+ }
+ }
+
+ /**
+* Decides the {@link RestartBackoffTimeStrategy} to use and its params
based on legacy configs,
+* and records its class name and the params into a adapted
configuration.
+*
+* The decision making is as follows:
+*
+* Use strategy of {@link
[GitHub] [flink] zhuzhurk commented on a change in pull request #8912: [FLINK-12709] [runtime] Implement new generation restart strategy loader which also respects legacy…
zhuzhurk commented on a change in pull request #8912: [FLINK-12709] [runtime]
Implement new generation restart strategy loader which also respects legacy…
URL: https://github.com/apache/flink/pull/8912#discussion_r324024299
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/RestartBackoffTimeStrategyFactoryLoader.java
##
@@ -0,0 +1,393 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.executiongraph.failover.flip1;
+
+import org.apache.flink.api.common.restartstrategy.RestartStrategies;
+import org.apache.flink.configuration.AkkaOptions;
+import org.apache.flink.configuration.ConfigConstants;
+import org.apache.flink.configuration.ConfigOption;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.configuration.RestartBackoffTimeStrategyOptions;
+import
org.apache.flink.runtime.executiongraph.restart.FixedDelayRestartStrategy;
+import org.apache.flink.runtime.executiongraph.restart.NoRestartStrategy;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.concurrent.TimeUnit;
+
+import scala.concurrent.duration.Duration;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * A utility class to load {@link RestartBackoffTimeStrategy.Factory} from the
configuration.
+ */
+public class RestartBackoffTimeStrategyFactoryLoader {
Review comment:
Ok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] GatsbyNewton commented on issue #9673: [FLINK-14031][examples][table]Added the blink planner dependency and …
GatsbyNewton commented on issue #9673: [FLINK-14031][examples][table]Added the blink planner dependency and … URL: https://github.com/apache/flink/pull/9673#issuecomment-531073836 Hi, @rmetzger. Could you please kindly help to review the code changes? Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124398771) * 6208617ff2d84bef7efaa7ee7cf96cba00031d88 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127279857) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #8859: [FLINK-12905][table-planner] Enable querying CatalogViews in legacy planner
xuefuz commented on a change in pull request #8859: [FLINK-12905][table-planner] Enable querying CatalogViews in legacy planner URL: https://github.com/apache/flink/pull/8859#discussion_r323991792 ## File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/catalog/AbstractCatalogView.java ## @@ -42,9 +43,15 @@ private final TableSchema schema; private final Map properties; private final String comment; + private final SqlDialect dialect; Review comment: This needs to be persisted somehow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #8859: [FLINK-12905][table-planner] Enable querying CatalogViews in legacy planner
xuefuz commented on a change in pull request #8859: [FLINK-12905][table-planner] Enable querying CatalogViews in legacy planner URL: https://github.com/apache/flink/pull/8859#discussion_r323991807 ## File path: flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/HiveCatalog.java ## @@ -534,6 +535,7 @@ private static CatalogBaseTable instantiateCatalogTable(Table hiveTable, HiveCon return new CatalogViewImpl( hiveTable.getViewOriginalText(), hiveTable.getViewExpandedText(), + SqlDialect.HIVE, Review comment: View stored in Hive catalog can belong to either Hive Dialect or Flink dialect. This is marked by "is_generic" property. Thus, it's inaccurate to assume "Hive" dialect here. Also, the property and SqlDialect enum has some duplication in meaning. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #8859: [FLINK-12905][table-planner] Enable querying CatalogViews in legacy planner
xuefuz commented on a change in pull request #8859:
[FLINK-12905][table-planner] Enable querying CatalogViews in legacy planner
URL: https://github.com/apache/flink/pull/8859#discussion_r323991723
##
File path:
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/FlinkViewTable.java
##
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner;
+
+import org.apache.flink.table.api.SqlDialect;
+import org.apache.flink.table.api.TableConfig;
+import org.apache.flink.table.api.ValidationException;
+
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.impl.ViewTable;
+
+import java.util.List;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * Thin extension to {@link ViewTable} that performs {@link SqlDialect} check
+ * before expanding the view. It throws exception if the view's dialect is
different
+ * from configured in the {@link TableConfig}.
+ */
+public class FlinkViewTable extends ViewTable {
+ private final SqlDialect viewDialect;
+
+ public FlinkViewTable(
+ SqlDialect viewDialect,
+ RelProtoDataType rowType,
+ String viewSql,
+ List schemaPath,
+ List viewPath) {
+ super(null, rowType, viewSql, schemaPath, viewPath);
+ this.viewDialect = checkNotNull(viewDialect);
+ }
+
+ @Override
+ public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable
relOptTable) {
+ TableConfig tableConfig =
context.getCluster().getPlanner().getContext().unwrap(TableConfig.class);
+ SqlDialect sqlDialect = tableConfig.getSqlDialect();
+ if (!viewDialect.equals(sqlDialect)) {
Review comment:
It's fine if we reject Hive views if the current dialect isn't set to Hive.
However, we need to consider the case that a Hive view is created by other
tools, where the dialect info isn't specifically set.
Not sure if this should be covered in this PR, but Hive views created by
Flink needs to be understandable by other Hive-compatible tools. To do that,
cross-catalog views are not good for Hive. Catalog-prefix also needs to be
trimmed from the expanded query.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124398771) * 6208617ff2d84bef7efaa7ee7cf96cba00031d88 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/127279857) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9581: [FLINK-13864][streaming]: Modify the StreamingFileSink Builder interface to allow for easier subclassing of StreamingFileSink
flinkbot edited a comment on issue #9581: [FLINK-13864][streaming]: Modify the StreamingFileSink Builder interface to allow for easier subclassing of StreamingFileSink URL: https://github.com/apache/flink/pull/9581#issuecomment-526712737 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit df82b55379806ac8502ef92999a1abc7f9a0056b (Thu Sep 12 23:03:55 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13864).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124398771) * 6208617ff2d84bef7efaa7ee7cf96cba00031d88 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yxu-valleytider commented on issue #9581: [FLINK-13864][streaming]: Modify the StreamingFileSink Builder interface to allow for easier subclassing of StreamingFileSink
yxu-valleytider commented on issue #9581: [FLINK-13864][streaming]: Modify the StreamingFileSink Builder interface to allow for easier subclassing of StreamingFileSink URL: https://github.com/apache/flink/pull/9581#issuecomment-531038789 sorry for the delay @kl0u . Was side tracked by a few work items and will pick this up in the new a few days. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-493155027 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6208617ff2d84bef7efaa7ee7cf96cba00031d88 (Thu Sep 12 22:55:47 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @beyond1920, @godfreyhe, @kurtyoung * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
walterddr commented on a change in pull request #8468:
[FLINK-12399][table][table-planner] Fix FilterableTableSource does not change
after applyPredicate
URL: https://github.com/apache/flink/pull/8468#discussion_r323980865
##
File path:
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/plan/nodes/PhysicalTableSourceScan.scala
##
@@ -47,9 +47,25 @@ abstract class PhysicalTableSourceScan(
val terms = super.explainTerms(pw)
.item("fields", deriveRowType().getFieldNames.asScala.mkString(", "))
+val auxiliarySourceDesc = tableSource match {
+ case fts: FilterableTableSource[_] =>
+s"FilterPushDown=${fts.isFilterPushedDown.toString}"
+ case pts: ProjectableTableSource[_] =>
+// TODO: add isTableProjected, or getProjectedFieldIndices API to
explain pushdown.
Review comment:
there's no way to determine currently for `ProjectableTableSource`. I will
create a follow up ticket for this once the solution is accepted.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-493155027 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6208617ff2d84bef7efaa7ee7cf96cba00031d88 (Thu Sep 12 22:51:44 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @beyond1920, @godfreyhe, @kurtyoung * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529930381 ## CI report: * 5bfec29d38218b1bd5236163a7f2dd2571afa8b2 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/126616886) * 166b8777fcc896f3f6ef7008133af35dbb554204 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126619729) * 9a6afcb65af3c4c59381e77fa93b2df73d2f2216 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127150468) * 09b61ef86b135960b2d21c6cf8d5f510684137ad : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127152805) * 9441f984cf179d8dc9212ffc59aea4b5ef922350 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127155093) * 52720e056437080ecc2906a61d59283b709f61a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127195882) * 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127198668) * 53b62a899ea4b0f71012a780f674dcd04191ee0d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127217387) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14057) Add Remove Other Timers to TimerService
[ https://issues.apache.org/jira/browse/FLINK-14057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928708#comment-16928708 ] Jesse Anderson commented on FLINK-14057: Of the ideas put forth, I like Elias' the best. I would change it slightly to replaceAllProcessingTimeTimers to reinforce that everything else gets removed. > Add Remove Other Timers to TimerService > --- > > Key: FLINK-14057 > URL: https://issues.apache.org/jira/browse/FLINK-14057 > Project: Flink > Issue Type: Improvement >Reporter: Jesse Anderson >Priority: Major > > The TimerService service has the ability to add timers with > registerProcessingTimeTimer. This method can be called many times and have > different timer times. > If you want to add a new timer and delete other timers, you have to keep > track of all previous timer times and call deleteProcessingTimeTimer for each > time. This method forces you to keep track of all previous (unexpired) timers > for a key. > Instead, I suggest overloading registerProcessingTimeTimer with a second > boolean argument that will remove all previous timers and set the new timer. > Note: although I'm using registerProcessingTimeTimer, this applies to > registerEventTimeTimer as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13862) Remove or rewrite Execution Plan docs
[
https://issues.apache.org/jira/browse/FLINK-13862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928682#comment-16928682
]
Stephan Ewen commented on FLINK-13862:
--
Is that plan visualizer something we want to keep supporting?
The new Web UI is pretty decent at the visualization and I would guess subsumes
it for the majority of users.
> Remove or rewrite Execution Plan docs
> -
>
> Key: FLINK-13862
> URL: https://issues.apache.org/jira/browse/FLINK-13862
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.9.0
>Reporter: Stephan Ewen
>Priority: Blocker
> Fix For: 1.10.0, 1.9.1
>
>
> The *Execution Plans* section is totally outdated and refers to the old
> {{tools/planVisalizer.html}} file that has been removed for two years.
> https://ci.apache.org/projects/flink/flink-docs-master/dev/execution_plans.html
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-14057) Add Remove Other Timers to TimerService
[
https://issues.apache.org/jira/browse/FLINK-14057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928676#comment-16928676
]
Elias Levy commented on FLINK-14057:
Could also add {{replaceProcessingTimeTimer}} and {{replaceEventTimeTimer}} .
> Add Remove Other Timers to TimerService
> ---
>
> Key: FLINK-14057
> URL: https://issues.apache.org/jira/browse/FLINK-14057
> Project: Flink
> Issue Type: Improvement
>Reporter: Jesse Anderson
>Priority: Major
>
> The TimerService service has the ability to add timers with
> registerProcessingTimeTimer. This method can be called many times and have
> different timer times.
> If you want to add a new timer and delete other timers, you have to keep
> track of all previous timer times and call deleteProcessingTimeTimer for each
> time. This method forces you to keep track of all previous (unexpired) timers
> for a key.
> Instead, I suggest overloading registerProcessingTimeTimer with a second
> boolean argument that will remove all previous timers and set the new timer.
> Note: although I'm using registerProcessingTimeTimer, this applies to
> registerEventTimeTimer as well.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529930381 ## CI report: * 5bfec29d38218b1bd5236163a7f2dd2571afa8b2 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/126616886) * 166b8777fcc896f3f6ef7008133af35dbb554204 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126619729) * 9a6afcb65af3c4c59381e77fa93b2df73d2f2216 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127150468) * 09b61ef86b135960b2d21c6cf8d5f510684137ad : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127152805) * 9441f984cf179d8dc9212ffc59aea4b5ef922350 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127155093) * 52720e056437080ecc2906a61d59283b709f61a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127195882) * 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127198668) * 53b62a899ea4b0f71012a780f674dcd04191ee0d : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/127217387) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14072) Add a third-party maven repository for flink-shaded .
[ https://issues.apache.org/jira/browse/FLINK-14072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928668#comment-16928668 ] Jeff Yang commented on FLINK-14072: --- The [PR|https://github.com/apache/flink-shaded/pull/71]. > Add a third-party maven repository for flink-shaded . > - > > Key: FLINK-14072 > URL: https://issues.apache.org/jira/browse/FLINK-14072 > Project: Flink > Issue Type: Bug > Components: BuildSystem / Shaded >Affects Versions: 1.9.0, 1.10.0 >Reporter: Jeff Yang >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0, 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Add a third-party maven repository to avoid the package being found when > custom compiling. Such as CDH,HDP,MapR. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-14072) Add a third-party maven repository for flink-shaded .
[ https://issues.apache.org/jira/browse/FLINK-14072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928666#comment-16928666 ] Jeff Yang commented on FLINK-14072: --- Hi,[~Zentol] [~sunjincheng121] Please take a look, Thank you very much. > Add a third-party maven repository for flink-shaded . > - > > Key: FLINK-14072 > URL: https://issues.apache.org/jira/browse/FLINK-14072 > Project: Flink > Issue Type: Bug > Components: BuildSystem / Shaded >Affects Versions: 1.9.0, 1.10.0 >Reporter: Jeff Yang >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0, 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Add a third-party maven repository to avoid the package being found when > custom compiling. Such as CDH,HDP,MapR. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (FLINK-14072) Add a third-party maven repository for flink-shaded .
[ https://issues.apache.org/jira/browse/FLINK-14072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-14072: --- Labels: pull-request-available (was: ) > Add a third-party maven repository for flink-shaded . > - > > Key: FLINK-14072 > URL: https://issues.apache.org/jira/browse/FLINK-14072 > Project: Flink > Issue Type: Bug > Components: BuildSystem / Shaded >Affects Versions: 1.9.0, 1.10.0 >Reporter: Jeff Yang >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0, 1.10.0 > > > Add a third-party maven repository to avoid the package being found when > custom compiling. Such as CDH,HDP,MapR. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529930381 ## CI report: * 5bfec29d38218b1bd5236163a7f2dd2571afa8b2 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/126616886) * 166b8777fcc896f3f6ef7008133af35dbb554204 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126619729) * 9a6afcb65af3c4c59381e77fa93b2df73d2f2216 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127150468) * 09b61ef86b135960b2d21c6cf8d5f510684137ad : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127152805) * 9441f984cf179d8dc9212ffc59aea4b5ef922350 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127155093) * 52720e056437080ecc2906a61d59283b709f61a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127195882) * 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127198668) * 53b62a899ea4b0f71012a780f674dcd04191ee0d : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api
flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api URL: https://github.com/apache/flink/pull/9013#issuecomment-514125271 ## CI report: * 2d2908752122665dc6bc45ceb4aa28099755f8d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120146040) * dbf5079d7ce8b8710758fb00fe2ab6727ebe72ad : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120200569) * 097af583c8040c16da17b796f6e4060c270b5b1d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120308925) * 38bdd2d9210071e951634b9171b33807f5d6b198 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127204619) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14072) Add a third-party maven repository for flink-shaded .
Jeff Yang created FLINK-14072: - Summary: Add a third-party maven repository for flink-shaded . Key: FLINK-14072 URL: https://issues.apache.org/jira/browse/FLINK-14072 Project: Flink Issue Type: Bug Components: BuildSystem / Shaded Affects Versions: 1.9.0, 1.10.0 Reporter: Jeff Yang Fix For: 1.10.0, 1.9.0 Add a third-party maven repository to avoid the package being found when custom compiling. Such as CDH,HDP,MapR. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 53b62a899ea4b0f71012a780f674dcd04191ee0d (Thu Sep 12 15:06:22 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9668: [hotfix][docs] Fix the wrong characters in the document(building.md, building.zh.md)
flinkbot edited a comment on issue #9668: [hotfix][docs] Fix the wrong characters in the document(building.md,building.zh.md) URL: https://github.com/apache/flink/pull/9668#issuecomment-530213212 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5d39c66f65f649f710b7d6100ca10fa2874da5ce (Thu Sep 12 15:01:15 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yangjf2019 commented on issue #9668: [hotfix][docs] Fix the wrong characters in the document(building.md, building.zh.md)
yangjf2019 commented on issue #9668: [hotfix][docs] Fix the wrong characters in the document(building.md,building.zh.md) URL: https://github.com/apache/flink/pull/9668#issuecomment-530866521 Hi, @wuchong Delay your time, please take a look, thank you very much! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13025) Elasticsearch 7.x support
[ https://issues.apache.org/jira/browse/FLINK-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928593#comment-16928593 ] Leonid Ilyevsky commented on FLINK-13025: - [~StephanEwen] sure, will do. > Elasticsearch 7.x support > - > > Key: FLINK-13025 > URL: https://issues.apache.org/jira/browse/FLINK-13025 > Project: Flink > Issue Type: New Feature > Components: Connectors / ElasticSearch >Affects Versions: 1.8.0 >Reporter: Keegan Standifer >Priority: Major > > Elasticsearch 7.0.0 was released in April of 2019: > [https://www.elastic.co/blog/elasticsearch-7-0-0-released] > The latest elasticsearch connector is > [flink-connector-elasticsearch6|https://github.com/apache/flink/tree/master/flink-connectors/flink-connector-elasticsearch6] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529930381 ## CI report: * 5bfec29d38218b1bd5236163a7f2dd2571afa8b2 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/126616886) * 166b8777fcc896f3f6ef7008133af35dbb554204 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126619729) * 9a6afcb65af3c4c59381e77fa93b2df73d2f2216 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127150468) * 09b61ef86b135960b2d21c6cf8d5f510684137ad : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127152805) * 9441f984cf179d8dc9212ffc59aea4b5ef922350 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127155093) * 52720e056437080ecc2906a61d59283b709f61a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127195882) * 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127198668) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api
flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api URL: https://github.com/apache/flink/pull/9013#issuecomment-514125271 ## CI report: * 2d2908752122665dc6bc45ceb4aa28099755f8d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120146040) * dbf5079d7ce8b8710758fb00fe2ab6727ebe72ad : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120200569) * 097af583c8040c16da17b796f6e4060c270b5b1d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120308925) * 38bdd2d9210071e951634b9171b33807f5d6b198 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/127204619) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api
flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api URL: https://github.com/apache/flink/pull/9013#issuecomment-514125271 ## CI report: * 2d2908752122665dc6bc45ceb4aa28099755f8d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120146040) * dbf5079d7ce8b8710758fb00fe2ab6727ebe72ad : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120200569) * 097af583c8040c16da17b796f6e4060c270b5b1d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120308925) * 38bdd2d9210071e951634b9171b33807f5d6b198 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13862) Remove or rewrite Execution Plan docs
[
https://issues.apache.org/jira/browse/FLINK-13862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928568#comment-16928568
]
Gary Yao commented on FLINK-13862:
--
{{planVisualizer.html}} was just a redirect to
https://flink.apache.org/visualizer/ so technically it still exists.
See
https://github.com/apache/flink/blob/e1b0e0bb7e4715a4315e6d8367a70276dbb8eb7a/flink-dist/src/main/flink-bin/tools/planVisualizer.html#L27
> Remove or rewrite Execution Plan docs
> -
>
> Key: FLINK-13862
> URL: https://issues.apache.org/jira/browse/FLINK-13862
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.9.0
>Reporter: Stephan Ewen
>Priority: Blocker
> Fix For: 1.10.0, 1.9.1
>
>
> The *Execution Plans* section is totally outdated and refers to the old
> {{tools/planVisalizer.html}} file that has been removed for two years.
> https://ci.apache.org/projects/flink/flink-docs-master/dev/execution_plans.html
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (FLINK-14066) pyflink building failure in master and 1.9.0 version
[
https://issues.apache.org/jira/browse/FLINK-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Yao updated FLINK-14066:
-
Component/s: API / Python
> pyflink building failure in master and 1.9.0 version
>
>
> Key: FLINK-14066
> URL: https://issues.apache.org/jira/browse/FLINK-14066
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Build System
>Affects Versions: 1.9.0, 1.10.0
> Environment: windows 10 enterprise x64（mentioned as build
> environment, not development environment.）
> powershell x64
> flink source master and 1.9.0 version
> jdk-8u202
> maven-3.2.5
>Reporter: Xu Yang
>Priority: Blocker
> Labels: beginner, build
> Attachments: setup.py
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ATTENTION: This is a issue about building pyflink, not development.
> During we build pyflink...
> After we have built flink from flink source code, a folder named "target" is
> generated.
> Then, following the document description, "cd flink-python; python3 setup.py
> sdist bdist_wheel", error happens.
> Root cause: in the setup.py file, line 75, "FLINK_HOME =
> os.path.abspath("../build-target")", the program can't found folder
> "build-target", however, the building of flink generated a folder named
> "target". So error happens in this way...
>
> The right way:
> in ../flink-python/setup.py line 75, modify code as following:
> FLINK_HOME = os.path.abspath("../target")
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529930381 ## CI report: * 5bfec29d38218b1bd5236163a7f2dd2571afa8b2 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/126616886) * 166b8777fcc896f3f6ef7008133af35dbb554204 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126619729) * 9a6afcb65af3c4c59381e77fa93b2df73d2f2216 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127150468) * 09b61ef86b135960b2d21c6cf8d5f510684137ad : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127152805) * 9441f984cf179d8dc9212ffc59aea4b5ef922350 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127155093) * 52720e056437080ecc2906a61d59283b709f61a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127195882) * 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/127198668) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 (Thu Sep 12 14:01:06 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323756671
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultScheduler.java
##
@@ -75,10 +137,281 @@ public DefaultScheduler(
slotRequestTimeout,
shuffleMaster,
partitionTracker);
+
+ this.restartBackoffTimeStrategy =
checkNotNull(restartBackoffTimeStrategy);
+ this.slotRequestTimeout = checkNotNull(slotRequestTimeout);
+ this.slotProvider = checkNotNull(slotProvider);
+ this.delayExecutor = checkNotNull(delayExecutor);
+ this.userCodeLoader = checkNotNull(userCodeLoader);
+ this.schedulingStrategyFactory =
checkNotNull(schedulingStrategyFactory);
+ this.failoverStrategyFactory =
checkNotNull(failoverStrategyFactory);
+ this.executionVertexOperations =
checkNotNull(executionVertexOperations);
+ this.executionVertexVersioner =
checkNotNull(executionVertexVersioner);
+ this.conditionalFutureHandlerFactory = new
ConditionalFutureHandlerFactory(executionVertexVersioner);
}
+ //
+ // SchedulerNG
+ //
+
@Override
- public void startScheduling() {
- throw new UnsupportedOperationException();
+ public void startSchedulingInternal() {
+ initializeScheduling();
+ schedulingStrategy.startScheduling();
+ }
+
+ private void initializeScheduling() {
+ executionFailureHandler = new
ExecutionFailureHandler(failoverStrategyFactory.create(getFailoverTopology()),
restartBackoffTimeStrategy);
+ schedulingStrategy =
schedulingStrategyFactory.createInstance(this, getSchedulingTopology(),
getJobGraph());
+ executionSlotAllocator = new
DefaultExecutionSlotAllocator(slotProvider, getInputsLocationsRetriever(),
slotRequestTimeout);
+ setTaskFailureListener(new
UpdateTaskExecutionStateInDefaultSchedulerListener(this,
getJobGraph().getJobID()));
+ prepareExecutionGraphForScheduling();
+ }
+
+ @Override
+ public boolean updateTaskExecutionState(final TaskExecutionState
taskExecutionState) {
+ final Optional executionVertexIdOptional =
getExecutionVertexId(taskExecutionState.getID());
+ if (executionVertexIdOptional.isPresent()) {
+ final ExecutionVertexID executionVertexId =
executionVertexIdOptional.get();
+ updateState(taskExecutionState);
+
schedulingStrategy.onExecutionStateChange(executionVertexId,
taskExecutionState.getExecutionState());
+ maybeHandleTaskFailure(taskExecutionState,
executionVertexId);
+ return true;
+ }
+
+ return false;
+ }
+
+ private void maybeHandleTaskFailure(final TaskExecutionState
taskExecutionState, final ExecutionVertexID executionVertexId) {
+ if (taskExecutionState.getExecutionState() ==
ExecutionState.FAILED) {
+ final Throwable error =
taskExecutionState.getError(userCodeLoader);
+ handleTaskFailure(executionVertexId, error);
+ }
+ }
+
+ private void handleTaskFailure(final ExecutionVertexID
executionVertexId, final Throwable error) {
+ final FailureHandlingResult failureHandlingResult =
executionFailureHandler.getFailureHandlingResult(executionVertexId, error);
+ maybeRestartTasks(failureHandlingResult);
+ }
+
+ private void maybeRestartTasks(final FailureHandlingResult
failureHandlingResult) {
+ if (failureHandlingResult.canRestart()) {
+ restartTasksWithDelay(failureHandlingResult);
+ } else {
+ failJob(failureHandlingResult.getError());
+ }
+ }
+
+ private void restartTasksWithDelay(final FailureHandlingResult
failureHandlingResult) {
+ final Set verticesToRestart =
failureHandlingResult.getVerticesToRestart();
+
+ final Set executionVertexVersions =
+ new
HashSet<>(executionVertexVersioner.recordVertexModifications(verticesToRestart).values());
+
+ final CompletableFuture cancelFuture =
cancelTasksAsync(verticesToRestart);
+
+ delayExecutor.schedule(
+ () -> FutureUtils.assertNoException(
+
cancelFuture.handleAsync(restartTasksOrHandleError(executionVertexVersions),
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 (Thu Sep 12 13:58:02 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhuzhurk commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
zhuzhurk commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#discussion_r323755152 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultSchedulerFactory.java ## @@ -23,11 +23,19 @@ import org.apache.flink.configuration.Configuration; import org.apache.flink.runtime.blob.BlobWriter; import org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory; +import org.apache.flink.runtime.concurrent.ScheduledExecutorServiceAdapter; +import org.apache.flink.runtime.executiongraph.failover.flip1.NeverRestartBackoffTimeStrategy; +import org.apache.flink.runtime.executiongraph.failover.flip1.RestartBackoffTimeStrategy; +import org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionStrategy; Review comment: Agreed. I think we shall rename them when removing the legacy restart strategies and failover strategies in 1.11(maybe). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-513425405 ## CI report: * bd7ade5e0b57dc8577d7f864afcbbb24c2513e56 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119869757) * 6a187929b931a4bd8cd7dbd0ec3d2c5a7a98278d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121411219) * 4f2afd322f96aeaba6d9c0b67a82a051eff22df0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121723032) * c4fc6905d3adf3ad9ff6f58c5d4f472fdfa7d52b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121724039) * b1855d5dfff586e41f152a6861ae04f30042cfde : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127193220) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529930381 ## CI report: * 5bfec29d38218b1bd5236163a7f2dd2571afa8b2 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/126616886) * 166b8777fcc896f3f6ef7008133af35dbb554204 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126619729) * 9a6afcb65af3c4c59381e77fa93b2df73d2f2216 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127150468) * 09b61ef86b135960b2d21c6cf8d5f510684137ad : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127152805) * 9441f984cf179d8dc9212ffc59aea4b5ef922350 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127155093) * 52720e056437080ecc2906a61d59283b709f61a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127195882) * 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8dfe15d8ea104f097077c7aedeae2ea5f49aae60 (Thu Sep 12 13:39:44 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-513425405 ## CI report: * bd7ade5e0b57dc8577d7f864afcbbb24c2513e56 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119869757) * 6a187929b931a4bd8cd7dbd0ec3d2c5a7a98278d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121411219) * 4f2afd322f96aeaba6d9c0b67a82a051eff22df0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121723032) * c4fc6905d3adf3ad9ff6f58c5d4f472fdfa7d52b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121724039) * b1855d5dfff586e41f152a6861ae04f30042cfde : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/127193220) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529930381 ## CI report: * 5bfec29d38218b1bd5236163a7f2dd2571afa8b2 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/126616886) * 166b8777fcc896f3f6ef7008133af35dbb554204 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126619729) * 9a6afcb65af3c4c59381e77fa93b2df73d2f2216 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127150468) * 09b61ef86b135960b2d21c6cf8d5f510684137ad : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/127152805) * 9441f984cf179d8dc9212ffc59aea4b5ef922350 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127155093) * 52720e056437080ecc2906a61d59283b709f61a5 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14066) pyflink building failure in master and 1.9.0 version
[
https://issues.apache.org/jira/browse/FLINK-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928522#comment-16928522
]
Xu Yang commented on FLINK-14066:
-
Right now, I only focus on the building stage of pyflink.
The application is running on liunx or docker.
I think I will take time to commit new setup.py next week if no one argues
other opinions.
> pyflink building failure in master and 1.9.0 version
>
>
> Key: FLINK-14066
> URL: https://issues.apache.org/jira/browse/FLINK-14066
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.9.0, 1.10.0
> Environment: windows 10 enterprise x64（mentioned as build
> environment, not development environment.）
> powershell x64
> flink source master and 1.9.0 version
> jdk-8u202
> maven-3.2.5
>Reporter: Xu Yang
>Priority: Blocker
> Labels: beginner, build
> Attachments: setup.py
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ATTENTION: This is a issue about building pyflink, not development.
> During we build pyflink...
> After we have built flink from flink source code, a folder named "target" is
> generated.
> Then, following the document description, "cd flink-python; python3 setup.py
> sdist bdist_wheel", error happens.
> Root cause: in the setup.py file, line 75, "FLINK_HOME =
> os.path.abspath("../build-target")", the program can't found folder
> "build-target", however, the building of flink generated a folder named
> "target". So error happens in this way...
>
> The right way:
> in ../flink-python/setup.py line 75, modify code as following:
> FLINK_HOME = os.path.abspath("../target")
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 52720e056437080ecc2906a61d59283b709f61a5 (Thu Sep 12 13:25:28 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#discussion_r323737657 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultSchedulerFactory.java ## @@ -23,11 +23,19 @@ import org.apache.flink.configuration.Configuration; import org.apache.flink.runtime.blob.BlobWriter; import org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory; +import org.apache.flink.runtime.concurrent.ScheduledExecutorServiceAdapter; +import org.apache.flink.runtime.executiongraph.failover.flip1.NeverRestartBackoffTimeStrategy; +import org.apache.flink.runtime.executiongraph.failover.flip1.RestartBackoffTimeStrategy; +import org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionStrategy; Review comment: cc: @zhuzhurk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#discussion_r323737277 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultSchedulerFactory.java ## @@ -23,11 +23,19 @@ import org.apache.flink.configuration.Configuration; import org.apache.flink.runtime.blob.BlobWriter; import org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory; +import org.apache.flink.runtime.concurrent.ScheduledExecutorServiceAdapter; +import org.apache.flink.runtime.executiongraph.failover.flip1.NeverRestartBackoffTimeStrategy; +import org.apache.flink.runtime.executiongraph.failover.flip1.RestartBackoffTimeStrategy; +import org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionStrategy; Review comment: Duly noted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 52720e056437080ecc2906a61d59283b709f61a5 (Thu Sep 12 13:23:26 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323735412
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionGraph.java
##
@@ -942,10 +950,28 @@ public void attachJobGraph(List
topologiallySorted) throws JobExcepti
new DefaultFailoverTopology(this));
}
+ public void setLegacyScheduling(final boolean legacyScheduling) {
Review comment:
I think it's possible but because we create the `ExecutionGraph` in
`SchedulerBase`, it would be also awkward to pass this information up to
`SchedulerBase` from the subclasses.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 7c5af1b8d7bfd440d8151c638a398d52fe9979af (Thu Sep 12 13:21:25 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323735412
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionGraph.java
##
@@ -942,10 +950,28 @@ public void attachJobGraph(List
topologiallySorted) throws JobExcepti
new DefaultFailoverTopology(this));
}
+ public void setLegacyScheduling(final boolean legacyScheduling) {
Review comment:
I think it's possible but because we create the `ExecutionGraph` in
`SchedulerBase`, it would be also awkward to pass this information to
`SchedulerBase` from the subclasses.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-14034) In FlinkKafkaProducer, KafkaTransactionState should be made public or invoke should be made final
[ https://issues.apache.org/jira/browse/FLINK-14034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928515#comment-16928515 ] Stephan Ewen commented on FLINK-14034: -- I would suggest to go with the variant that does not extend the sink (extra function). The sink is bound to change in the future, so the other variant would probably also be more safe against future changes in Flink. > In FlinkKafkaProducer, KafkaTransactionState should be made public or invoke > should be made final > - > > Key: FLINK-14034 > URL: https://issues.apache.org/jira/browse/FLINK-14034 > Project: Flink > Issue Type: Wish > Components: Connectors / Kafka >Affects Versions: 1.9.0 >Reporter: Niels van Kaam >Priority: Trivial > > It is not possible to override the invoke method of the FlinkKafkaProducer, > because the first parameter, KafkaTransactionState, is a private inner class. > It is not possible to override the original invoke of SinkFunction, because > TwoPhaseCommitSinkFunction (which is implemented by FlinkKafkaProducer) does > override the original invoke method with final. > [https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer.java] > If there is a particular reason for this, it would be better to make the > invoke method in FlinkKafkaProducer final as well, and document the reason > such that it is clear this is by design (I don't see any overrides in the > same package). > Otherwise, I would make the KafkaTransactionState publicly visible. I would > like to override the Invoke method to create a custom KafkaProducer which > performs some additional generic validations and transformations. (which can > also be done in a process-function, but a custom sink would simplify the code > of jobs) > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323591961
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/FailingExecutionVertexOperationsDecorator.java
##
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.runtime.scheduler;
+
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.concurrent.FutureUtils;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+
+import java.util.concurrent.CompletableFuture;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * Allows to fail ExecutionVertex operations for testing.
+ */
+public class FailingExecutionVertexOperationsDecorator implements
ExecutionVertexOperations {
+
+ private final ExecutionVertexOperations delegate;
+
+ private boolean failDeploy;
+
+ private boolean failCancel;
+
+ public FailingExecutionVertexOperationsDecorator(final
ExecutionVertexOperations delegate) {
+ this.delegate = checkNotNull(delegate);
+ }
+
+ @Override
+ public void deploy(final ExecutionVertex executionVertex, final
DeploymentOption deploymentOption) throws JobException {
+ if (failDeploy) {
+ throw new RuntimeException("Expected");
+ } else {
+ delegate.deploy(executionVertex, deploymentOption);
+ }
+ }
+
+ @Override
+ public CompletableFuture cancel(final ExecutionVertex
executionVertex) {
+ if (failCancel) {
+ return FutureUtils.completedExceptionally(new
RuntimeException("Expected"));
+ } else {
+ return delegate.cancel(executionVertex);
+ }
+ }
+
+ public void enableFailDeploy() {
+ failDeploy = true;
+ }
+
+ public void disableFailDeploy() {
+ failDeploy = false;
+ }
+
+ public void enableFailCancel() {
+ failCancel = true;
+ }
+
+ public void disableFailCancel() {
+ failCancel = false;
+ }
Review comment:
I don't understand. Do you want to declare `failCancel` `final` or set the
default value to `false` (which is already the case)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-14002) FlinkKafkaProducer constructor that takes KafkaSerializationSchema shouldnt take default topic
[ https://issues.apache.org/jira/browse/FLINK-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928513#comment-16928513 ] Gyula Fora commented on FLINK-14002: I am bit busy in the next few days but I can definitely fix this before the next release if no one picks it up earlier :) > FlinkKafkaProducer constructor that takes KafkaSerializationSchema shouldnt > take default topic > -- > > Key: FLINK-14002 > URL: https://issues.apache.org/jira/browse/FLINK-14002 > Project: Flink > Issue Type: Improvement > Components: Connectors / Kafka >Reporter: Gyula Fora >Priority: Major > > When the KafkaSerializationSchema is used the user has the to provide the > topic always when they create the ProducerRecord. > The defaultTopic specified in the constructor (and enforced not to be null) > will always be ignored, this is very misleading. > We should depracate these constructors and create new ones without > defaultTopic. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323591961
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/FailingExecutionVertexOperationsDecorator.java
##
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.runtime.scheduler;
+
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.concurrent.FutureUtils;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+
+import java.util.concurrent.CompletableFuture;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * Allows to fail ExecutionVertex operations for testing.
+ */
+public class FailingExecutionVertexOperationsDecorator implements
ExecutionVertexOperations {
+
+ private final ExecutionVertexOperations delegate;
+
+ private boolean failDeploy;
+
+ private boolean failCancel;
+
+ public FailingExecutionVertexOperationsDecorator(final
ExecutionVertexOperations delegate) {
+ this.delegate = checkNotNull(delegate);
+ }
+
+ @Override
+ public void deploy(final ExecutionVertex executionVertex, final
DeploymentOption deploymentOption) throws JobException {
+ if (failDeploy) {
+ throw new RuntimeException("Expected");
+ } else {
+ delegate.deploy(executionVertex, deploymentOption);
+ }
+ }
+
+ @Override
+ public CompletableFuture cancel(final ExecutionVertex
executionVertex) {
+ if (failCancel) {
+ return FutureUtils.completedExceptionally(new
RuntimeException("Expected"));
+ } else {
+ return delegate.cancel(executionVertex);
+ }
+ }
+
+ public void enableFailDeploy() {
+ failDeploy = true;
+ }
+
+ public void disableFailDeploy() {
+ failDeploy = false;
+ }
+
+ public void enableFailCancel() {
+ failCancel = true;
+ }
+
+ public void disableFailCancel() {
+ failCancel = false;
+ }
Review comment:
I don't understand. Do you want to declare `failCancel` `final`, or do you
want to set the default value to `false` (which is already the case)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates
flinkbot edited a comment on issue #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates URL: https://github.com/apache/flink/pull/9670#issuecomment-530272387 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 124081df8ab34a71754d67c751dc87f321b6d1bf (Thu Sep 12 13:17:21 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 7c5af1b8d7bfd440d8151c638a398d52fe9979af (Thu Sep 12 13:16:19 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] asfgit closed pull request #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates
asfgit closed pull request #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates URL: https://github.com/apache/flink/pull/9670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#discussion_r323732826 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/LegacyScheduler.java ## @@ -19,78 +19,24 @@ package org.apache.flink.runtime.scheduler; -import org.apache.flink.api.common.JobID; -import org.apache.flink.api.common.restartstrategy.RestartStrategies; import org.apache.flink.api.common.time.Time; -import org.apache.flink.configuration.CheckpointingOptions; import org.apache.flink.configuration.Configuration; -import org.apache.flink.core.io.InputSplit; -import org.apache.flink.queryablestate.KvStateID; -import org.apache.flink.runtime.JobException; -import org.apache.flink.runtime.accumulators.AccumulatorSnapshot; import org.apache.flink.runtime.blob.BlobWriter; -import org.apache.flink.runtime.checkpoint.CheckpointCoordinator; -import org.apache.flink.runtime.checkpoint.CheckpointMetrics; import org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory; -import org.apache.flink.runtime.checkpoint.CompletedCheckpoint; -import org.apache.flink.runtime.checkpoint.TaskStateSnapshot; -import org.apache.flink.runtime.client.JobExecutionException; -import org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor; -import org.apache.flink.runtime.concurrent.FutureUtils; -import org.apache.flink.runtime.execution.ExecutionState; -import org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph; -import org.apache.flink.runtime.executiongraph.Execution; -import org.apache.flink.runtime.executiongraph.ExecutionAttemptID; import org.apache.flink.runtime.executiongraph.ExecutionGraph; -import org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder; -import org.apache.flink.runtime.executiongraph.ExecutionGraphException; -import org.apache.flink.runtime.executiongraph.ExecutionJobVertex; -import org.apache.flink.runtime.executiongraph.IntermediateResult; -import org.apache.flink.runtime.executiongraph.JobStatusListener; -import org.apache.flink.runtime.executiongraph.restart.RestartStrategy; import org.apache.flink.runtime.executiongraph.restart.RestartStrategyFactory; -import org.apache.flink.runtime.executiongraph.restart.RestartStrategyResolving; import org.apache.flink.runtime.io.network.partition.PartitionTracker; -import org.apache.flink.runtime.io.network.partition.ResultPartitionID; -import org.apache.flink.runtime.jobgraph.IntermediateDataSetID; import org.apache.flink.runtime.jobgraph.JobGraph; -import org.apache.flink.runtime.jobgraph.JobStatus; -import org.apache.flink.runtime.jobgraph.JobVertexID; -import org.apache.flink.runtime.jobgraph.SavepointRestoreSettings; -import org.apache.flink.runtime.jobmanager.PartitionProducerDisposedException; -import org.apache.flink.runtime.jobmaster.SerializedInputSplit; import org.apache.flink.runtime.jobmaster.slotpool.SlotProvider; -import org.apache.flink.runtime.messages.FlinkJobNotFoundException; -import org.apache.flink.runtime.messages.checkpoint.AcknowledgeCheckpoint; -import org.apache.flink.runtime.messages.checkpoint.DeclineCheckpoint; -import org.apache.flink.runtime.messages.webmonitor.JobDetails; import org.apache.flink.runtime.metrics.groups.JobManagerJobMetricGroup; -import org.apache.flink.runtime.query.KvStateLocation; -import org.apache.flink.runtime.query.KvStateLocationRegistry; -import org.apache.flink.runtime.query.UnknownKvStateLocation; import org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureStatsTracker; -import org.apache.flink.runtime.rest.handler.legacy.backpressure.OperatorBackPressureStats; import org.apache.flink.runtime.shuffle.ShuffleMaster; -import org.apache.flink.runtime.state.KeyGroupRange; -import org.apache.flink.runtime.taskmanager.TaskExecutionState; -import org.apache.flink.runtime.taskmanager.TaskManagerLocation; -import org.apache.flink.runtime.webmonitor.WebMonitorUtils; -import org.apache.flink.util.FlinkException; -import org.apache.flink.util.InstantiationUtil; -import org.apache.flink.util.function.FunctionUtils; import org.slf4j.Logger; -import java.io.IOException; -import java.net.InetSocketAddress; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; import java.util.concurrent.Executor; import java.util.concurrent.ScheduledExecutorService; -import static org.apache.flink.util.Preconditions.checkNotNull; Review comment: Cannot fix this without rebasing now. I will probably squash all commits into one anyways. (except for hotfixes) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at:
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#discussion_r323732826 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/LegacyScheduler.java ## @@ -19,78 +19,24 @@ package org.apache.flink.runtime.scheduler; -import org.apache.flink.api.common.JobID; -import org.apache.flink.api.common.restartstrategy.RestartStrategies; import org.apache.flink.api.common.time.Time; -import org.apache.flink.configuration.CheckpointingOptions; import org.apache.flink.configuration.Configuration; -import org.apache.flink.core.io.InputSplit; -import org.apache.flink.queryablestate.KvStateID; -import org.apache.flink.runtime.JobException; -import org.apache.flink.runtime.accumulators.AccumulatorSnapshot; import org.apache.flink.runtime.blob.BlobWriter; -import org.apache.flink.runtime.checkpoint.CheckpointCoordinator; -import org.apache.flink.runtime.checkpoint.CheckpointMetrics; import org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory; -import org.apache.flink.runtime.checkpoint.CompletedCheckpoint; -import org.apache.flink.runtime.checkpoint.TaskStateSnapshot; -import org.apache.flink.runtime.client.JobExecutionException; -import org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor; -import org.apache.flink.runtime.concurrent.FutureUtils; -import org.apache.flink.runtime.execution.ExecutionState; -import org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph; -import org.apache.flink.runtime.executiongraph.Execution; -import org.apache.flink.runtime.executiongraph.ExecutionAttemptID; import org.apache.flink.runtime.executiongraph.ExecutionGraph; -import org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder; -import org.apache.flink.runtime.executiongraph.ExecutionGraphException; -import org.apache.flink.runtime.executiongraph.ExecutionJobVertex; -import org.apache.flink.runtime.executiongraph.IntermediateResult; -import org.apache.flink.runtime.executiongraph.JobStatusListener; -import org.apache.flink.runtime.executiongraph.restart.RestartStrategy; import org.apache.flink.runtime.executiongraph.restart.RestartStrategyFactory; -import org.apache.flink.runtime.executiongraph.restart.RestartStrategyResolving; import org.apache.flink.runtime.io.network.partition.PartitionTracker; -import org.apache.flink.runtime.io.network.partition.ResultPartitionID; -import org.apache.flink.runtime.jobgraph.IntermediateDataSetID; import org.apache.flink.runtime.jobgraph.JobGraph; -import org.apache.flink.runtime.jobgraph.JobStatus; -import org.apache.flink.runtime.jobgraph.JobVertexID; -import org.apache.flink.runtime.jobgraph.SavepointRestoreSettings; -import org.apache.flink.runtime.jobmanager.PartitionProducerDisposedException; -import org.apache.flink.runtime.jobmaster.SerializedInputSplit; import org.apache.flink.runtime.jobmaster.slotpool.SlotProvider; -import org.apache.flink.runtime.messages.FlinkJobNotFoundException; -import org.apache.flink.runtime.messages.checkpoint.AcknowledgeCheckpoint; -import org.apache.flink.runtime.messages.checkpoint.DeclineCheckpoint; -import org.apache.flink.runtime.messages.webmonitor.JobDetails; import org.apache.flink.runtime.metrics.groups.JobManagerJobMetricGroup; -import org.apache.flink.runtime.query.KvStateLocation; -import org.apache.flink.runtime.query.KvStateLocationRegistry; -import org.apache.flink.runtime.query.UnknownKvStateLocation; import org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureStatsTracker; -import org.apache.flink.runtime.rest.handler.legacy.backpressure.OperatorBackPressureStats; import org.apache.flink.runtime.shuffle.ShuffleMaster; -import org.apache.flink.runtime.state.KeyGroupRange; -import org.apache.flink.runtime.taskmanager.TaskExecutionState; -import org.apache.flink.runtime.taskmanager.TaskManagerLocation; -import org.apache.flink.runtime.webmonitor.WebMonitorUtils; -import org.apache.flink.util.FlinkException; -import org.apache.flink.util.InstantiationUtil; -import org.apache.flink.util.function.FunctionUtils; import org.slf4j.Logger; -import java.io.IOException; -import java.net.InetSocketAddress; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; import java.util.concurrent.Executor; import java.util.concurrent.ScheduledExecutorService; -import static org.apache.flink.util.Preconditions.checkNotNull; Review comment: Cannot fix this without rebasing. I will probably squash all commits into one anyways. (except for hotfixes) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on issue #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates
flinkbot edited a comment on issue #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates URL: https://github.com/apache/flink/pull/9670#issuecomment-530272387 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 124081df8ab34a71754d67c751dc87f321b6d1bf (Thu Sep 12 13:15:17 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-513425405 ## CI report: * bd7ade5e0b57dc8577d7f864afcbbb24c2513e56 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119869757) * 6a187929b931a4bd8cd7dbd0ec3d2c5a7a98278d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121411219) * 4f2afd322f96aeaba6d9c0b67a82a051eff22df0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121723032) * c4fc6905d3adf3ad9ff6f58c5d4f472fdfa7d52b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121724039) * b1855d5dfff586e41f152a6861ae04f30042cfde : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 7c5af1b8d7bfd440d8151c638a398d52fe9979af (Thu Sep 12 13:14:17 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323732451
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultScheduler.java
##
@@ -75,10 +137,281 @@ public DefaultScheduler(
slotRequestTimeout,
shuffleMaster,
partitionTracker);
+
+ this.restartBackoffTimeStrategy = restartBackoffTimeStrategy;
+ this.slotRequestTimeout = slotRequestTimeout;
+ this.slotProvider = slotProvider;
+ this.delayExecutor = delayExecutor;
+ this.userCodeLoader = userCodeLoader;
+ this.schedulingStrategyFactory =
checkNotNull(schedulingStrategyFactory);
+ this.failoverStrategyFactory =
checkNotNull(failoverStrategyFactory);
+ this.executionVertexOperations =
checkNotNull(executionVertexOperations);
+ this.executionVertexVersioner = executionVertexVersioner;
+ this.conditionalFutureHandlerFactory = new
ConditionalFutureHandlerFactory(executionVertexVersioner);
+ }
+
+ //
+ // SchedulerNG
+ //
+
+ @Override
+ public void startSchedulingInternal() {
+ initializeScheduling();
+ schedulingStrategy.startScheduling();
+ }
+
+ private void initializeScheduling() {
+ executionFailureHandler = new
ExecutionFailureHandler(failoverStrategyFactory.create(getFailoverTopology()),
restartBackoffTimeStrategy);
+ schedulingStrategy =
schedulingStrategyFactory.createInstance(this, getSchedulingTopology(),
getJobGraph());
+ executionSlotAllocator = new
DefaultExecutionSlotAllocator(slotProvider, getInputsLocationsRetriever(),
slotRequestTimeout);
+ setTaskFailureListener(new
UpdateTaskExecutionStateInDefaultSchedulerListener(this,
getJobGraph().getJobID()));
+ prepareExecutionGraphForScheduling();
+ }
+
+ @Override
+ public boolean updateTaskExecutionState(final TaskExecutionState
taskExecutionState) {
+ final Optional executionVertexIdOptional =
getExecutionVertexId(taskExecutionState.getID());
+ if (executionVertexIdOptional.isPresent()) {
+ final ExecutionVertexID executionVertexId =
executionVertexIdOptional.get();
+ updateState(taskExecutionState);
+
schedulingStrategy.onExecutionStateChange(executionVertexId,
taskExecutionState.getExecutionState());
+ maybeHandleTaskFailure(taskExecutionState,
executionVertexId);
+ return true;
+ }
+
+ return false;
+ }
+
+ private void maybeHandleTaskFailure(final TaskExecutionState
taskExecutionState, final ExecutionVertexID executionVertexId) {
+ if (taskExecutionState.getExecutionState() ==
ExecutionState.FAILED) {
+ final Throwable error =
taskExecutionState.getError(userCodeLoader);
+ handleTaskFailure(executionVertexId, error);
+ }
+ }
+
+ private void handleTaskFailure(final ExecutionVertexID
executionVertexId, final Throwable error) {
+ final FailureHandlingResult failureHandlingResult =
executionFailureHandler.getFailureHandlingResult(executionVertexId, error);
+ maybeRestartTasks(failureHandlingResult);
+ }
+
+ private void maybeRestartTasks(final FailureHandlingResult
failureHandlingResult) {
+ if (failureHandlingResult.canRestart()) {
+ restartTasksWithDelay(failureHandlingResult);
+ } else {
+ failJob(failureHandlingResult.getError());
+ }
+ }
+
+ private void restartTasksWithDelay(final FailureHandlingResult
failureHandlingResult) {
+ final Set verticesToRestart =
failureHandlingResult.getVerticesToRestart();
+
+ final Set executionVertexVersions =
+ new
HashSet<>(executionVertexVersioner.recordVertexModifications(verticesToRestart).values());
+
+ final CompletableFuture cancelFuture =
cancelTasksAsync(verticesToRestart);
+
+ delayExecutor.schedule(
+ () -> FutureUtils.assertNoException(
+
cancelFuture.handleAsync(restartTasksOrHandleError(executionVertexVersions),
getMainThreadExecutor())),
+ failureHandlingResult.getRestartDelayMS(),
+ TimeUnit.MILLISECONDS);
+ }
+
+ private BiFunction
[GitHub] [flink] StephanEwen commented on issue #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates
StephanEwen commented on issue #9670: [FLINK-14049] [coordination] Add task name to error message for failed partition updates URL: https://github.com/apache/flink/pull/9670#issuecomment-530819672 Thanks, merging this... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323731494
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultExecutionVertexOperations.java
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.runtime.scheduler;
+
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+
+import java.util.concurrent.CompletableFuture;
+
+class DefaultExecutionVertexOperations implements ExecutionVertexOperations {
+
+ @Override
+ public void deploy(final ExecutionVertex executionVertex, final
DeploymentOption deploymentOption) throws JobException {
+
executionVertex.setSendScheduleOrUpdateConsumerMessage(deploymentOption.sendScheduleOrUpdateConsumerMessage());
Review comment:
See 41dd6626222ccf237cea327c95c1c1100553561b for fix.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323731494
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultExecutionVertexOperations.java
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.runtime.scheduler;
+
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+
+import java.util.concurrent.CompletableFuture;
+
+class DefaultExecutionVertexOperations implements ExecutionVertexOperations {
+
+ @Override
+ public void deploy(final ExecutionVertex executionVertex, final
DeploymentOption deploymentOption) throws JobException {
+
executionVertex.setSendScheduleOrUpdateConsumerMessage(deploymentOption.sendScheduleOrUpdateConsumerMessage());
Review comment:
See 41dd6626222ccf237cea327c95c1c1100553561b for a fix.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323731494
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultExecutionVertexOperations.java
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.runtime.scheduler;
+
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+
+import java.util.concurrent.CompletableFuture;
+
+class DefaultExecutionVertexOperations implements ExecutionVertexOperations {
+
+ @Override
+ public void deploy(final ExecutionVertex executionVertex, final
DeploymentOption deploymentOption) throws JobException {
+
executionVertex.setSendScheduleOrUpdateConsumerMessage(deploymentOption.sendScheduleOrUpdateConsumerMessage());
Review comment:
See 41dd6626222ccf237cea327c95c1c1100553561b for the fix.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
flinkbot edited a comment on issue #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub URL: https://github.com/apache/flink/pull/9663#issuecomment-529928149 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 7c5af1b8d7bfd440d8151c638a398d52fe9979af (Thu Sep 12 13:12:14 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime] Implement DefaultScheduler stub
GJL commented on a change in pull request #9663: [WIP][FLINK-12433][runtime]
Implement DefaultScheduler stub
URL: https://github.com/apache/flink/pull/9663#discussion_r323731161
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionVertex.java
##
@@ -365,6 +368,14 @@ public InputDependencyConstraint
getInputDependencyConstraint() {
return getJobVertex().getInputDependencyConstraint();
}
+ public void setSendScheduleOrUpdateConsumerMessage(final boolean
sendScheduleOrUpdateConsumerMessage) {
+ this.sendScheduleOrUpdateConsumerMessage =
sendScheduleOrUpdateConsumerMessage;
+ }
+
+ public boolean isSendScheduleOrUpdateConsumerMessage() {
+ return sendScheduleOrUpdateConsumerMessage;
+ }
Review comment:
See 41dd6626222ccf237cea327c95c1c1100553561b for an alternative
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-13025) Elasticsearch 7.x support
[ https://issues.apache.org/jira/browse/FLINK-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928511#comment-16928511 ] Stephan Ewen commented on FLINK-13025: -- [~lilyevsky] would you be up to working closely together with [~yanghua] to give him feedback on the implementation, if [~yanghua] drives the implementation? > Elasticsearch 7.x support > - > > Key: FLINK-13025 > URL: https://issues.apache.org/jira/browse/FLINK-13025 > Project: Flink > Issue Type: New Feature > Components: Connectors / ElasticSearch >Affects Versions: 1.8.0 >Reporter: Keegan Standifer >Priority: Major > > Elasticsearch 7.0.0 was released in April of 2019: > [https://www.elastic.co/blog/elasticsearch-7-0-0-released] > The latest elasticsearch connector is > [flink-connector-elasticsearch6|https://github.com/apache/flink/tree/master/flink-connectors/flink-connector-elasticsearch6] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor
flinkbot edited a comment on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor URL: https://github.com/apache/flink/pull/9646#issuecomment-529228507 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 78fbc2d6e6a7275560caac22a26ae3865a7ead27 (Thu Sep 12 13:10:12 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor
StephanEwen commented on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor URL: https://github.com/apache/flink/pull/9646#issuecomment-530817767 I am also not sure why the runtime need to know about this interface. As far as I understand the current state of the FLIP-27 discussions, the `SourceReader` runs in a source operator (which handles the events, splits-to-checkpointed-state mapping, etc.). The runtime would interact only with that source operator. The fact that the source operator as a source reader internally seems like a detail that should not be leaking into the runtime. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor
flinkbot edited a comment on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor URL: https://github.com/apache/flink/pull/9646#issuecomment-529228507 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 78fbc2d6e6a7275560caac22a26ae3865a7ead27 (Thu Sep 12 13:08:11 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14066) pyflink building failure in master and 1.9.0 version
[
https://issues.apache.org/jira/browse/FLINK-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928507#comment-16928507
]
Dian Fu commented on FLINK-14066:
-
I mean that for windows support, we need to consider both building and running
pyflink on windows.
If you want to just build pyflink on windows(do not need to run it on windows),
then we can firstly fix the build issue. If you also have requirements to run
pyflink on windows, then we need also add the corresponding window scripts for
scripts such as pyflink-gateway-server.sh.
> pyflink building failure in master and 1.9.0 version
>
>
> Key: FLINK-14066
> URL: https://issues.apache.org/jira/browse/FLINK-14066
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.9.0, 1.10.0
> Environment: windows 10 enterprise x64（mentioned as build
> environment, not development environment.）
> powershell x64
> flink source master and 1.9.0 version
> jdk-8u202
> maven-3.2.5
>Reporter: Xu Yang
>Priority: Blocker
> Labels: beginner, build
> Attachments: setup.py
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ATTENTION: This is a issue about building pyflink, not development.
> During we build pyflink...
> After we have built flink from flink source code, a folder named "target" is
> generated.
> Then, following the document description, "cd flink-python; python3 setup.py
> sdist bdist_wheel", error happens.
> Root cause: in the setup.py file, line 75, "FLINK_HOME =
> os.path.abspath("../build-target")", the program can't found folder
> "build-target", however, the building of flink generated a folder named
> "target". So error happens in this way...
>
> The right way:
> in ../flink-python/setup.py line 75, modify code as following:
> FLINK_HOME = os.path.abspath("../target")
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (FLINK-14071) [for Documents Improvement] Add explicit path where jars files should be put for pyflink development
Xu Yang created FLINK-14071: --- Summary: [for Documents Improvement] Add explicit path where jars files should be put for pyflink development Key: FLINK-14071 URL: https://issues.apache.org/jira/browse/FLINK-14071 Project: Flink Issue Type: Improvement Components: API / Python Affects Versions: 1.9.0 Reporter: Xu Yang for docu improvement: I propose that Add explicit path where jars files should be put for pyflink development. for some beginners, they are on a hard way to run a demo successful, not to mention a production coding... When it comes to pyflink development, I found the docu lacks a explicit descritption of where the jars files should be put. The recommended path is: *../site-packages/pyflink/lib/your_jar_files* -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] StephanEwen commented on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor
StephanEwen commented on issue #9646: [FLINK-14004][task] Define SourceReader interface to verify the integration with StreamOneInputProcessor URL: https://github.com/apache/flink/pull/9646#issuecomment-530816761 I think in that case there should be a placeholder interface in `flink-runtime` that is used, instead of a preliminary version of `SourceReader`. That placeholder gets refactored/replaced by the proper source reader then. That placeholder should also have only the necessary methods, nothing about splits, start, etc. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-513424964 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c4fc6905d3adf3ad9ff6f58c5d4f472fdfa7d52b (Thu Sep 12 12:58:57 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13339).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add
an implementation of pipeline's api
URL: https://github.com/apache/flink/pull/9184#discussion_r323724380
##
File path:
flink-ml-parent/flink-ml-lib/src/main/java/org/apache/flink/ml/pipeline/Estimator.java
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.ml.pipeline;
+
+import org.apache.flink.ml.api.misc.param.Params;
+import org.apache.flink.ml.batchoperator.BatchOperator;
+import org.apache.flink.ml.batchoperator.source.TableSourceBatchOp;
+import org.apache.flink.ml.common.MLSession;
+import org.apache.flink.ml.streamoperator.StreamOperator;
+import org.apache.flink.ml.streamoperator.source.TableSourceStreamOp;
+import org.apache.flink.table.api.Table;
+import org.apache.flink.table.api.TableEnvironment;
+import org.apache.flink.table.api.internal.TableImpl;
+import org.apache.flink.table.api.java.StreamTableEnvironment;
+
+/**
+ * Abstract class for a estimator that fit a {@link Model}.
+ *
+ * @param The class type of the {@link Estimator} implementation itself
+ * @param class type of the {@link Model} this Estimator produces.
+ */
+public abstract class Estimator, M extends Model
>
+ extends PipelineStage implements
org.apache.flink.ml.api.core.Estimator {
+
+ public Estimator() {
+ super();
+ }
+
+ public Estimator(Params params) {
+ super(params);
+ }
+
+ /**
+* Train and produce a {@link Model} which fits the records in the
given {@link Table}.
Review comment:
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add
an implementation of pipeline's api
URL: https://github.com/apache/flink/pull/9184#discussion_r323724073
##
File path:
flink-ml-parent/flink-ml-lib/src/main/java/org/apache/flink/ml/common/AlgoOperator.java
##
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.ml.common;
+
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.ml.api.misc.param.Params;
+import org.apache.flink.ml.api.misc.param.WithParams;
+import org.apache.flink.ml.batchoperator.source.TableSourceBatchOp;
+import org.apache.flink.ml.streamoperator.source.TableSourceStreamOp;
+import org.apache.flink.table.api.Table;
+import org.apache.flink.table.api.TableSchema;
+import org.apache.flink.table.api.internal.TableImpl;
+import org.apache.flink.table.api.java.StreamTableEnvironment;
+
+import java.io.Serializable;
+
+/**
+ * Base class for algorithm operators.
+ * @param The class type of the {@link AlgoOperator} implementation itself
+ */
+public abstract class AlgoOperator> implements
WithParams, Serializable {
+
+ protected Params params;
Review comment:
Thanks for your advice, these fields are refactored to private.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-513424964 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c4fc6905d3adf3ad9ff6f58c5d4f472fdfa7d52b (Thu Sep 12 12:56:55 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13339).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add
an implementation of pipeline's api
URL: https://github.com/apache/flink/pull/9184#discussion_r323722921
##
File path:
flink-ml-parent/flink-ml-lib/src/main/java/org/apache/flink/ml/streamoperator/StreamOperator.java
##
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.ml.streamoperator;
+
+import org.apache.flink.ml.api.misc.param.Params;
+import org.apache.flink.ml.common.AlgoOperator;
+import org.apache.flink.ml.streamoperator.source.TableSourceStreamOp;
+import org.apache.flink.table.api.Table;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Base class of streaming algorithm operators.
+ */
+public abstract class StreamOperator> extends
AlgoOperator {
+
+ public StreamOperator() {
+ super();
+ }
+
+ public StreamOperator(Params params) {
+ super(params);
+ }
+
+ public static StreamOperator sourceFrom(Table table) {
+ return new TableSourceStreamOp(table);
+ }
+
+ @Override
+ public String toString() {
+ return getOutput().toString();
+ }
+
+ public S link(S next) {
Review comment:
link() and linkTo() return the next AlgoOp, linkFrom() returns self AlgoOp.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928499#comment-16928499 ] Stephan Ewen commented on FLINK-13417: -- @Tison would it be safer to shade Flink's ZK and then let the HBase client bring its own version / config / everything? I guess in a proper setup, it should work anyways due to inverted class loading (HBase connector uses its own dependency in the user jar, not the Flink ZK dependency), but not for tests. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Blocker > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-513424964 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c4fc6905d3adf3ad9ff6f58c5d4f472fdfa7d52b (Thu Sep 12 12:47:43 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13339).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuyang1706 edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-530808471 > Hi @xuyang1706, > thanks for your work, I've just left a few comments, please look it then you will have time. Hi @ex00 , thanks for your comments. We define the new method linkFrom(BatchOperator…), it can support one, two or many inputs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuyang1706 commented on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 commented on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-530808471 > Hi @xuyang1706, > thanks for your work, I've just left a few comments, please look it then you will have time. Thanks for your comments. We define the new method linkFrom(BatchOperator…), it can support one, two or many inputs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add
an implementation of pipeline's api
URL: https://github.com/apache/flink/pull/9184#discussion_r323717999
##
File path:
flink-ml-parent/flink-ml-lib/src/main/java/org/apache/flink/ml/batchoperator/BatchOperator.java
##
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.ml.batchoperator;
+
+import org.apache.flink.ml.api.misc.param.Params;
+import org.apache.flink.ml.batchoperator.source.TableSourceBatchOp;
+import org.apache.flink.ml.common.AlgoOperator;
+import org.apache.flink.table.api.Table;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Base class of batch algorithm operators.
+ */
+public abstract class BatchOperator> extends
AlgoOperator {
+
+ public BatchOperator() {
+ super();
+ }
+
+ public BatchOperator(Params params) {
+ super(params);
+ }
+
+ public static BatchOperator sourceFrom(Table table) {
+ return new TableSourceBatchOp(table);
+ }
+
+ @Override
+ public String toString() {
+ return getOutput().toString();
+ }
+
+ public B link(B next) {
+ return linkTo(next);
+ }
+
+ public B linkTo(B next) {
+ next.linkFrom(this);
+ return (B) next;
+ }
+
+ public abstract T linkFrom(BatchOperator in);
+
+ public T linkFrom(BatchOperator in1, BatchOperator in2) {
+ List ls = new ArrayList();
+ ls.add(in1);
+ ls.add(in2);
+ return linkFrom(ls);
+ }
+
+ public T linkFrom(BatchOperator in1, BatchOperator in2, BatchOperator
in3) {
+ List ls = new ArrayList();
+ ls.add(in1);
+ ls.add(in2);
+ ls.add(in3);
+ return linkFrom(ls);
+ }
+
+ public T linkFrom(List ins) {
+ if (null != ins && ins.size() == 1) {
+ return linkFrom(ins.get(0));
+ } else {
+ throw new RuntimeException("Not support more than 1
inputs!");
Review comment:
We define the new method linkFrom(BatchOperator…) to replace them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
flinkbot edited a comment on issue #9184: [FLINK-13339][ml] Add an implementation of pipeline's api URL: https://github.com/apache/flink/pull/9184#issuecomment-513424964 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c4fc6905d3adf3ad9ff6f58c5d4f472fdfa7d52b (Thu Sep 12 12:44:41 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13339).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add
an implementation of pipeline's api
URL: https://github.com/apache/flink/pull/9184#discussion_r323718017
##
File path:
flink-ml-parent/flink-ml-lib/src/main/java/org/apache/flink/ml/batchoperator/BatchOperator.java
##
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.ml.batchoperator;
+
+import org.apache.flink.ml.api.misc.param.Params;
+import org.apache.flink.ml.batchoperator.source.TableSourceBatchOp;
+import org.apache.flink.ml.common.AlgoOperator;
+import org.apache.flink.table.api.Table;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Base class of batch algorithm operators.
+ */
+public abstract class BatchOperator> extends
AlgoOperator {
+
+ public BatchOperator() {
+ super();
+ }
+
+ public BatchOperator(Params params) {
+ super(params);
+ }
+
+ public static BatchOperator sourceFrom(Table table) {
+ return new TableSourceBatchOp(table);
+ }
+
+ @Override
+ public String toString() {
+ return getOutput().toString();
+ }
+
+ public B link(B next) {
+ return linkTo(next);
+ }
+
+ public B linkTo(B next) {
+ next.linkFrom(this);
+ return (B) next;
+ }
+
+ public abstract T linkFrom(BatchOperator in);
+
+ public T linkFrom(BatchOperator in1, BatchOperator in2) {
+ List ls = new ArrayList();
+ ls.add(in1);
+ ls.add(in2);
+ return linkFrom(ls);
+ }
+
+ public T linkFrom(BatchOperator in1, BatchOperator in2, BatchOperator
in3) {
+ List ls = new ArrayList();
+ ls.add(in1);
+ ls.add(in2);
+ ls.add(in3);
+ return linkFrom(ls);
+ }
+
+ public T linkFrom(List ins) {
+ if (null != ins && ins.size() == 1) {
Review comment:
We define a new method linkFrom(BatchOperator…) to replace both of them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (FLINK-14066) pyflink building failure in master and 1.9.0 version
[
https://issues.apache.org/jira/browse/FLINK-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Yang updated FLINK-14066:
Summary: pyflink building failure in master and 1.9.0 version (was: bug of
building pyflink in master and 1.9.0 version)
> pyflink building failure in master and 1.9.0 version
>
>
> Key: FLINK-14066
> URL: https://issues.apache.org/jira/browse/FLINK-14066
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.9.0, 1.10.0
> Environment: windows 10 enterprise x64（mentioned as build
> environment, not development environment.）
> powershell x64
> flink source master and 1.9.0 version
> jdk-8u202
> maven-3.2.5
>Reporter: Xu Yang
>Priority: Blocker
> Labels: beginner, build
> Attachments: setup.py
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ATTENTION: This is a issue about building pyflink, not development.
> During we build pyflink...
> After we have built flink from flink source code, a folder named "target" is
> generated.
> Then, following the document description, "cd flink-python; python3 setup.py
> sdist bdist_wheel", error happens.
> Root cause: in the setup.py file, line 75, "FLINK_HOME =
> os.path.abspath("../build-target")", the program can't found folder
> "build-target", however, the building of flink generated a folder named
> "target". So error happens in this way...
>
> The right way:
> in ../flink-python/setup.py line 75, modify code as following:
> FLINK_HOME = os.path.abspath("../target")
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-14066) bug of building pyflink in master and 1.9.0 version
[
https://issues.apache.org/jira/browse/FLINK-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928492#comment-16928492
]
Xu Yang commented on FLINK-14066:
-
Dear Dian Fu, tks for watching my issue. I know pyflink is not supported for
development on windows. All my work is about the building stage of pyflink.
The issue I report is about the building stage of pyflink.
I am a little confused about your comment, do you mean the
"pyflink-gateway-server.sh" that can't be run on win is the root cause of
building pyflink failure？
> bug of building pyflink in master and 1.9.0 version
> ---
>
> Key: FLINK-14066
> URL: https://issues.apache.org/jira/browse/FLINK-14066
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.9.0, 1.10.0
> Environment: windows 10 enterprise x64（mentioned as build
> environment, not development environment.）
> powershell x64
> flink source master and 1.9.0 version
> jdk-8u202
> maven-3.2.5
>Reporter: Xu Yang
>Priority: Blocker
> Labels: beginner, build
> Attachments: setup.py
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ATTENTION: This is a issue about building pyflink, not development.
> During we build pyflink...
> After we have built flink from flink source code, a folder named "target" is
> generated.
> Then, following the document description, "cd flink-python; python3 setup.py
> sdist bdist_wheel", error happens.
> Root cause: in the setup.py file, line 75, "FLINK_HOME =
> os.path.abspath("../build-target")", the program can't found folder
> "build-target", however, the building of flink generated a folder named
> "target". So error happens in this way...
>
> The right way:
> in ../flink-python/setup.py line 75, modify code as following:
> FLINK_HOME = os.path.abspath("../target")
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (FLINK-14066) bug of building pyflink in master and 1.9.0 version
[
https://issues.apache.org/jira/browse/FLINK-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Yang updated FLINK-14066:
Description:
ATTENTION: This is a issue about building pyflink, not development.
During we build pyflink...
After we have built flink from flink source code, a folder named "target" is
generated.
Then, following the document description, "cd flink-python; python3 setup.py
sdist bdist_wheel", error happens.
Root cause: in the setup.py file, line 75, "FLINK_HOME =
os.path.abspath("../build-target")", the program can't found folder
"build-target", however, the building of flink generated a folder named
"target". So error happens in this way...
The right way:
in ../flink-python/setup.py line 75, modify code as following:
FLINK_HOME = os.path.abspath("../target")
was:
During we build pyflink...
After we have built flink from flink source code, a folder named "target" is
generated.
Then, following the document description, "cd flink-python; python3 setup.py
sdist bdist_wheel", error happens.
Root cause: in the setup.py file, line 75, "FLINK_HOME =
os.path.abspath("../build-target")", the program can't found folder
"build-target", however, the building of flink generated a folder named
"target". So error happens in this way...
The right way:
in ../flink-python/setup.py line 75, modify code as following:
FLINK_HOME = os.path.abspath("../target")
> bug of building pyflink in master and 1.9.0 version
> ---
>
> Key: FLINK-14066
> URL: https://issues.apache.org/jira/browse/FLINK-14066
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.9.0, 1.10.0
> Environment: windows 10 enterprise x64（mentioned as build
> environment, not development environment.）
> powershell x64
> flink source master and 1.9.0 version
> jdk-8u202
> maven-3.2.5
>Reporter: Xu Yang
>Priority: Blocker
> Labels: beginner, build
> Attachments: setup.py
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ATTENTION: This is a issue about building pyflink, not development.
> During we build pyflink...
> After we have built flink from flink source code, a folder named "target" is
> generated.
> Then, following the document description, "cd flink-python; python3 setup.py
> sdist bdist_wheel", error happens.
> Root cause: in the setup.py file, line 75, "FLINK_HOME =
> os.path.abspath("../build-target")", the program can't found folder
> "build-target", however, the building of flink generated a folder named
> "target". So error happens in this way...
>
> The right way:
> in ../flink-python/setup.py line 75, modify code as following:
> FLINK_HOME = os.path.abspath("../target")
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (FLINK-14066) bug of building pyflink in master and 1.9.0 version
[
https://issues.apache.org/jira/browse/FLINK-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Yang updated FLINK-14066:
Environment:
windows 10 enterprise x64（mentioned as build environment, not development
environment.）
powershell x64
flink source master and 1.9.0 version
jdk-8u202
maven-3.2.5
was:
windows 10 enterprise x64
powershell x64
flink source master and 1.9.0 version
jdk-8u202
maven-3.2.5
> bug of building pyflink in master and 1.9.0 version
> ---
>
> Key: FLINK-14066
> URL: https://issues.apache.org/jira/browse/FLINK-14066
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.9.0, 1.10.0
> Environment: windows 10 enterprise x64（mentioned as build
> environment, not development environment.）
> powershell x64
> flink source master and 1.9.0 version
> jdk-8u202
> maven-3.2.5
>Reporter: Xu Yang
>Priority: Blocker
> Labels: beginner, build
> Attachments: setup.py
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> During we build pyflink...
> After we have built flink from flink source code, a folder named "target" is
> generated.
> Then, following the document description, "cd flink-python; python3 setup.py
> sdist bdist_wheel", error happens.
> Root cause: in the setup.py file, line 75, "FLINK_HOME =
> os.path.abspath("../build-target")", the program can't found folder
> "build-target", however, the building of flink generated a folder named
> "target". So error happens in this way...
>
> The right way:
> in ../flink-python/setup.py line 75, modify code as following:
> FLINK_HOME = os.path.abspath("../target")
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api
flinkbot edited a comment on issue #9013: [FLINK-13136] Fix documentation error about stopping job with restful api URL: https://github.com/apache/flink/pull/9013#issuecomment-509065631 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 097af583c8040c16da17b796f6e4060c270b5b1d (Thu Sep 12 12:06:00 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on a change in pull request #9013: [FLINK-13136] Fix documentation error about stopping job with restful api
GJL commented on a change in pull request #9013: [FLINK-13136] Fix
documentation error about stopping job with restful api
URL: https://github.com/apache/flink/pull/9013#discussion_r323702619
##
File path:
flink-clients/src/main/java/org/apache/flink/client/program/rest/RestClusterClient.java
##
@@ -375,7 +374,7 @@ public JobSubmissionResult submitJob(JobGraph jobGraph,
ClassLoader classLoader)
public void cancel(JobID jobID) throws Exception {
JobCancellationMessageParameters params = new
JobCancellationMessageParameters();
params.jobPathParameter.resolve(jobID);
-
params.terminationModeQueryParameter.resolve(Collections.singletonList(TerminationModeQueryParameter.TerminationMode.CANCEL));
+
params.terminationModeQueryParameter.resolve(Collections.singletonList("CANCEL"));
Review comment:
Ok thanks. I will wait for your changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Comment Edited] (FLINK-14070) Use TimeUtils to parse duration configs
[ https://issues.apache.org/jira/browse/FLINK-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928476#comment-16928476 ] Till Rohrmann edited comment on FLINK-14070 at 9/12/19 11:55 AM: - Nice improvement proposal [~zhuzh]. I've assigned you to work on it. was (Author: till.rohrmann): Nice improvement [~zhuzh]. I've assigned you to work on it. > Use TimeUtils to parse duration configs > --- > > Key: FLINK-14070 > URL: https://issues.apache.org/jira/browse/FLINK-14070 > Project: Flink > Issue Type: Improvement > Components: Runtime / Configuration >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Assignee: Zhu Zhu >Priority: Major > Fix For: 1.10.0 > > > FLINK-14069 makes TimeUtils able to parse all time unit labels supported by > scala Duration. > We can now use TimeUtils to parse duration configs instead of using scala > Duration. > Some config descriptors referring scala FiniteDuration should be updated as > well. > This is one step for Flink core to get rid of scala dependencies. -- This message was sent by Atlassian Jira (v8.3.2#803003)
