[GitHub] [flink] hameizi commented on pull request #15564: [FLINK-22207][connectors/hive]Hive Catalog retrieve Flink Properties …
hameizi commented on pull request #15564: URL: https://github.com/apache/flink/pull/15564#issuecomment-819246948 @KurtYoung I fix in the new commit, please review thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15601: [FLINK-18071][FLINK-21996][coordination] - Part two: Ensure reliable OperatorEvent to running Task matching
flinkbot edited a comment on pull request #15601: URL: https://github.com/apache/flink/pull/15601#issuecomment-819126497 ## CI report: * 6e5b9772808314793e666e5e2b9f2953dad30edd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16486) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15545: [FLINK-22167][table-planner] Partial insert not works when complex fields reorder
flinkbot edited a comment on pull request #15545: URL: https://github.com/apache/flink/pull/15545#issuecomment-816526358 ## CI report: * 55e3eb5247e7f6c44ef4e6bef04c34c8fc6af348 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16276) * 33ca6135b4141071ca8030e42de1d75b66561d3b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16501) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] kezhuw commented on pull request #15557: [FLINK-21996][coordination] - Part one: Tests and adjusted threading model
kezhuw commented on pull request #15557: URL: https://github.com/apache/flink/pull/15557#issuecomment-819230272 @StephanEwen HI, all my concerns are either solved or answered. From my side, it is in a merge-able state. It still be good and helpful if @tillrohrmann and/or @becketqin could give another eyes on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] kezhuw commented on a change in pull request #15557: [FLINK-21996][coordination] - Part one: Tests and adjusted threading model
kezhuw commented on a change in pull request #15557:
URL: https://github.com/apache/flink/pull/15557#discussion_r612937053
##
File path:
flink-tests/src/test/java/org/apache/flink/runtime/operators/coordination/OperatorEventSendingCheckpointITCase.java
##
@@ -0,0 +1,449 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.operators.coordination;
+
+import org.apache.flink.api.common.eventtime.WatermarkStrategy;
+import org.apache.flink.api.common.functions.MapFunction;
+import org.apache.flink.api.connector.source.SplitEnumerator;
+import org.apache.flink.api.connector.source.SplitEnumeratorContext;
+import org.apache.flink.api.connector.source.lib.NumberSequenceSource;
+import org.apache.flink.api.connector.source.lib.util.IteratorSourceEnumerator;
+import org.apache.flink.api.connector.source.lib.util.IteratorSourceSplit;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.akka.AkkaUtils;
+import org.apache.flink.runtime.concurrent.FutureUtils;
+import org.apache.flink.runtime.executiongraph.ExecutionAttemptID;
+import org.apache.flink.runtime.jobgraph.OperatorID;
+import org.apache.flink.runtime.messages.Acknowledge;
+import org.apache.flink.runtime.minicluster.MiniCluster;
+import org.apache.flink.runtime.minicluster.MiniClusterConfiguration;
+import org.apache.flink.runtime.minicluster.RpcServiceSharing;
+import org.apache.flink.runtime.rpc.RpcGateway;
+import org.apache.flink.runtime.rpc.RpcService;
+import org.apache.flink.runtime.rpc.akka.AkkaRpcService;
+import org.apache.flink.runtime.rpc.akka.AkkaRpcServiceConfiguration;
+import org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils;
+import org.apache.flink.runtime.source.event.AddSplitEvent;
+import org.apache.flink.runtime.source.event.NoMoreSplitsEvent;
+import org.apache.flink.runtime.taskexecutor.TaskExecutorGateway;
+import org.apache.flink.runtime.taskexecutor.TaskExecutorGatewayAdapter;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.util.TestStreamEnvironment;
+import org.apache.flink.util.SerializedValue;
+import org.apache.flink.util.function.TriFunction;
+
+import akka.actor.ActorSystem;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+import org.junit.Test;
+
+import javax.annotation.Nullable;
+
+import java.time.Duration;
+import java.util.ArrayDeque;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import java.util.stream.LongStream;
+
+import static org.junit.Assert.assertEquals;
+
+/**
+ * A test suite for source enumerator (operator coordinator) for situations
where RPC calls for
+ * split assignments (operator events) fails from time to time.
+ */
+@SuppressWarnings("serial")
+public class OperatorEventSendingCheckpointITCase {
+
+private static final int PARALLELISM = 1;
+private static MiniCluster flinkCluster;
+
+@BeforeClass
+public static void setupMiniClusterAndEnv() throws Exception {
+flinkCluster = new MiniClusterWithRpcIntercepting(PARALLELISM);
+flinkCluster.start();
+TestStreamEnvironment.setAsContext(flinkCluster, PARALLELISM);
+}
+
+@AfterClass
+public static void clearEnvAndStopMiniCluster() throws Exception {
+TestStreamEnvironment.unsetAsContext();
+if (flinkCluster != null) {
+flinkCluster.close();
+flinkCluster = null;
+}
+}
+
+//
+// tests
+//
+
+/**
+ * Every second assign split event is lost. Eventually, the enumerator
must recognize that an
+ * event was lost and trigger recovery to prevent data loss. Data loss
would manifest in a
+ * stalled test, because we could wait
[GitHub] [flink] flinkbot edited a comment on pull request #15602: [FLINK-22264][docs] Fix misleading statement about Flink Job Cluster Kubernetes Support in Flink Architecture page
flinkbot edited a comment on pull request #15602: URL: https://github.com/apache/flink/pull/15602#issuecomment-819181043 ## CI report: * d6c10842849e1d47bb035ac4ee140395f9214f3b UNKNOWN * a63a44d02327232df329255b5e5ab40484238b92 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16493) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15545: [FLINK-22167][table-planner] Partial insert not works when complex fields reorder
flinkbot edited a comment on pull request #15545: URL: https://github.com/apache/flink/pull/15545#issuecomment-816526358 ## CI report: * 55e3eb5247e7f6c44ef4e6bef04c34c8fc6af348 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16276) * 33ca6135b4141071ca8030e42de1d75b66561d3b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320694#comment-17320694
]
hehuiyuan commented on FLINK-22263:
---
[~lzljs3620320] ok.
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
> Fix For: 1.13.0
>
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
[jira] [Comment Edited] (FLINK-22047) Could not find FLINSHED Flink job and can't submit job
[ https://issues.apache.org/jira/browse/FLINK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320693#comment-17320693 ] hayden zhou edited comment on FLINK-22047 at 4/14/21, 4:30 AM: --- [~trohrmann] thanks for your reply, hopes to see the fixed version release as soon as possible:)(y)(y) was (Author: hayden zhou): [~trohrmann] thanks for your reply, hopes to see the fixed version release as soon as possible > Could not find FLINSHED Flink job and can't submit job > --- > > Key: FLINK-22047 > URL: https://issues.apache.org/jira/browse/FLINK-22047 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.2 >Reporter: hayden zhou >Assignee: Caizhi Weng >Priority: Major > Attachments: screenshot-1.png > > > Could not find FLINSHED Flink job, and aways can't submit job by > insufficient slot -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22047) Could not find FLINSHED Flink job and can't submit job
[ https://issues.apache.org/jira/browse/FLINK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320693#comment-17320693 ] hayden zhou commented on FLINK-22047: - [~trohrmann] thanks for your reply, hopes to see the fixed version release as soon as possible > Could not find FLINSHED Flink job and can't submit job > --- > > Key: FLINK-22047 > URL: https://issues.apache.org/jira/browse/FLINK-22047 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.2 >Reporter: hayden zhou >Assignee: Caizhi Weng >Priority: Major > Attachments: screenshot-1.png > > > Could not find FLINSHED Flink job, and aways can't submit job by > insufficient slot -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320691#comment-17320691
]
Jingsong Lee commented on FLINK-22263:
--
You can re-open FLINK-22021 and create a PR for 1.12.
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
> Fix For: 1.13.0
>
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
>
[jira] [Commented] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320690#comment-17320690
]
Jingsong Lee commented on FLINK-22263:
--
Yes, you can cherry-pick this to 1.12.
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
> Fix For: 1.13.0
>
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
[GitHub] [flink] flinkbot edited a comment on pull request #15603: [FLINK-22169][sql-client] Beautify the CliTableauResultView when print
flinkbot edited a comment on pull request #15603: URL: https://github.com/apache/flink/pull/15603#issuecomment-819212209 ## CI report: * a230294e88fd9cef432d1d53d41a163318efcf5d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16498) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15604: [FLINK-22265][javadocs]Abnormal document display
flinkbot edited a comment on pull request #15604: URL: https://github.com/apache/flink/pull/15604#issuecomment-819212247 ## CI report: * 8ca0e185d9016d9a0b4debafb90b1f59d8bdcee4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16499) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15605: [FLINK-21996][coordination] - Part 3&4: Ensure OperatorEvent transport losses are handled
flinkbot edited a comment on pull request #15605: URL: https://github.com/apache/flink/pull/15605#issuecomment-819212302 ## CI report: * d2111f6d5a17e9263399aac7c188083c793b418d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16500) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15557: [FLINK-21996][coordination] - Part one: Tests and adjusted threading model
flinkbot edited a comment on pull request #15557: URL: https://github.com/apache/flink/pull/15557#issuecomment-817412419 ## CI report: * 9e2711b0c5f1adaa5e561c7c6cf7c5b8647b7aeb Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16484) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15533: [FLINK-22063][table] Fix bug that some join conditions are lost when …
flinkbot edited a comment on pull request #15533: URL: https://github.com/apache/flink/pull/15533#issuecomment-816380956 ## CI report: * d4454ba96d5ca1b821b16192cd669eb87be74de7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16453) * d5d2090eed0386b0fd1ff953ec7bc5aef0d2cd72 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16497) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] JingsongLi commented on a change in pull request #15545: [FLINK-22167][table-planner] Partial insert not works when complex fields reorder
JingsongLi commented on a change in pull request #15545:
URL: https://github.com/apache/flink/pull/15545#discussion_r612925447
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/calcite/PreValidateReWriter.scala
##
@@ -272,7 +272,7 @@ object PreValidateReWriter {
targetPosition: util.List[Int]): util.ArrayList[SqlNode] = {
val targetList = new Array[SqlNode](sourceList.size())
0 until sourceList.size() foreach {
- idx => targetList(targetPosition.get(idx)) = sourceList.get(idx)
+ idx => targetList(idx) = sourceList.get(targetPosition.get(idx))
Review comment:
Actually, here whole method is `targetPosition.map(sourceList.get)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320680#comment-17320680
]
hehuiyuan commented on FLINK-22263:
---
Hi [~lzljs3620320] , why not merge to 1.12 ?
If this can be merged, I can do it.
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
> Fix For: 1.13.0
>
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
>
[GitHub] [flink] flinkbot commented on pull request #15605: [FLINK-21996][coordination] - Part 3&4: Ensure OperatorEvent transport losses are handled
flinkbot commented on pull request #15605: URL: https://github.com/apache/flink/pull/15605#issuecomment-819212302 ## CI report: * d2111f6d5a17e9263399aac7c188083c793b418d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15604: [FLINK-22265][javadocs]Abnormal document display
flinkbot commented on pull request #15604: URL: https://github.com/apache/flink/pull/15604#issuecomment-819212247 ## CI report: * 8ca0e185d9016d9a0b4debafb90b1f59d8bdcee4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15603: [FLINK-22169][sql-client] Beautify the CliTableauResultView when print
flinkbot commented on pull request #15603: URL: https://github.com/apache/flink/pull/15603#issuecomment-819212209 ## CI report: * a230294e88fd9cef432d1d53d41a163318efcf5d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15533: [FLINK-22063][table] Fix bug that some join conditions are lost when …
flinkbot edited a comment on pull request #15533: URL: https://github.com/apache/flink/pull/15533#issuecomment-816380956 ## CI report: * d4454ba96d5ca1b821b16192cd669eb87be74de7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16453) * d5d2090eed0386b0fd1ff953ec7bc5aef0d2cd72 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20633) test_configuration.test_add_all test failed in py35
[
https://issues.apache.org/jira/browse/FLINK-20633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320676#comment-17320676
]
Guowei Ma commented on FLINK-20633:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16480=logs=3e4dd1a2-fe2f-5e5d-a581-48087e718d53=45a89cfc-9ff2-5909-6443-6c732efcf06b=20665
> test_configuration.test_add_all test failed in py35
> ---
>
> Key: FLINK-20633
> URL: https://issues.apache.org/jira/browse/FLINK-20633
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0, 1.13.0
>Reporter: Huang Xingbo
>Priority: Major
> Labels: test-stability
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10946=logs=821b528f-1eed-5598-a3b4-7f748b13f261=4fad9527-b9a5-5015-1b70-8356e5c91490]
> {code:java}
> 2020-12-17T01:07:04.9062839Z ___
> ConfigurationTests.test_add_all
> 2020-12-17T01:07:04.9063107Z
> 2020-12-17T01:07:04.9063436Z self =
> testMethod=test_add_all>
> 2020-12-17T01:07:04.9063719Z
> 2020-12-17T01:07:04.9063951Z def test_add_all(self):
> 2020-12-17T01:07:04.9064224Z > conf = Configuration()
> 2020-12-17T01:07:04.9064411Z
> 2020-12-17T01:07:04.9064665Z pyflink/common/tests/test_configuration.py:85:
> 2020-12-17T01:07:04.9065074Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2020-12-17T01:07:04.9065474Z pyflink/common/configuration.py:43: in __init__
> 2020-12-17T01:07:04.9065765Z gateway = get_gateway()
> 2020-12-17T01:07:04.9066065Z pyflink/java_gateway.py:62: in get_gateway
> 2020-12-17T01:07:04.9066352Z _gateway = launch_gateway()
> 2020-12-17T01:07:04.9066671Z pyflink/java_gateway.py:104: in launch_gateway
> 2020-12-17T01:07:04.9076442Z p = launch_gateway_server_process(env, args)
> 2020-12-17T01:07:04.9076987Z pyflink/pyflink_gateway_server.py:197: in
> launch_gateway_server_process
> 2020-12-17T01:07:04.9079207Z download_apache_avro()
> 2020-12-17T01:07:04.9079558Z pyflink/pyflink_gateway_server.py:129: in
> download_apache_avro
> 2020-12-17T01:07:04.9163662Z cwd=flink_source_root).decode("utf-8")
> 2020-12-17T01:07:04.9164205Z
> dev/.conda/envs/3.5/lib/python3.5/subprocess.py:316: in check_output
> 2020-12-17T01:07:04.9164558Z **kwargs).stdout
> 2020-12-17T01:07:04.9164887Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2020-12-17T01:07:04.9165168Z
> 2020-12-17T01:07:04.9165396Z input = None, timeout = None, check = True
> 2020-12-17T01:07:04.9166036Z popenargs = (['mvn', 'help:evaluate',
> '-Dexpression=avro.version'],)
> 2020-12-17T01:07:04.9166615Z kwargs = {'cwd': '/__w/1/s', 'stdout': -1}
> 2020-12-17T01:07:04.9166959Z process = 0x7f0ff8a7a320>
> 2020-12-17T01:07:04.9168169Z stdout = b"[INFO] Scanning for
> projects...\nDownloading:
> https://repo.maven.apache.org/maven2/org/apache/apache/20/apache-20.po...R]
> After correcting the problems, you can resume the build with the
> command\n[ERROR] mvn -rf :flink-parent\n"
> 2020-12-17T01:07:04.9169079Z stderr = None, retcode = 1
> 2020-12-17T01:07:04.9169259Z
> 2020-12-17T01:07:04.9169600Z def run(*popenargs, input=None,
> timeout=None, check=False, **kwargs):
> 2020-12-17T01:07:04.9170061Z """Run command with arguments and return
> a CompletedProcess instance.
> 2020-12-17T01:07:04.9170373Z
> 2020-12-17T01:07:04.9170683Z The returned instance will have
> attributes args, returncode, stdout and
> 2020-12-17T01:07:04.9171117Z stderr. By default, stdout and stderr
> are not captured, and those attributes
> 2020-12-17T01:07:04.9171577Z will be None. Pass stdout=PIPE and/or
> stderr=PIPE in order to capture them.
> 2020-12-17T01:07:04.9171895Z
> 2020-12-17T01:07:04.9172409Z If check is True and the exit code was
> non-zero, it raises a
> 2020-12-17T01:07:04.9172852Z CalledProcessError. The
> CalledProcessError object will have the return code
> 2020-12-17T01:07:04.9173305Z in the returncode attribute, and output
> & stderr attributes if those streams
> 2020-12-17T01:07:04.9173662Z were captured.
> 2020-12-17T01:07:04.9173879Z
> 2020-12-17T01:07:04.9174175Z If timeout is given, and the process
> takes too long, a TimeoutExpired
> 2020-12-17T01:07:04.9174537Z exception will be raised.
> 2020-12-17T01:07:04.9174773Z
> 2020-12-17T01:07:04.9175040Z There is an optional argument "input",
> allowing you to
> 2020-12-17T01:07:04.9175663Z pass a string to the subprocess's stdin.
> If you use this argument
> 2020-12-17T01:07:04.9176301Z you may not also use the Popen
> constructor's "stdin" argument, as
>
[jira] [Commented] (FLINK-20254) HiveTableSourceITCase.testStreamPartitionReadByCreateTime times out
[
https://issues.apache.org/jira/browse/FLINK-20254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320675#comment-17320675
]
Guowei Ma commented on FLINK-20254:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16482=logs=8fd975ef-f478-511d-4997-6f15fe8a1fd3=ac0fa443-5d45-5a6b-3597-0310ecc1d2ab=25368
> HiveTableSourceITCase.testStreamPartitionReadByCreateTime times out
> ---
>
> Key: FLINK-20254
> URL: https://issues.apache.org/jira/browse/FLINK-20254
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
>Affects Versions: 1.12.0, 1.13.0
>Reporter: Robert Metzger
>Assignee: Leonard Xu
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.13.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=9808=logs=fc5181b0-e452-5c8f-68de-1097947f6483=62110053-334f-5295-a0ab-80dd7e2babbf
> {code}
> 2020-11-19T10:34:23.5591765Z [ERROR] Tests run: 18, Failures: 0, Errors: 1,
> Skipped: 0, Time elapsed: 192.243 s <<< FAILURE! - in
> org.apache.flink.connectors.hive.HiveTableSourceITCase
> 2020-11-19T10:34:23.5593193Z [ERROR]
> testStreamPartitionReadByCreateTime(org.apache.flink.connectors.hive.HiveTableSourceITCase)
> Time elapsed: 120.075 s <<< ERROR!
> 2020-11-19T10:34:23.5593929Z org.junit.runners.model.TestTimedOutException:
> test timed out after 12 milliseconds
> 2020-11-19T10:34:23.5594321Z at java.lang.Thread.sleep(Native Method)
> 2020-11-19T10:34:23.5594777Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.sleepBeforeRetry(CollectResultFetcher.java:231)
> 2020-11-19T10:34:23.5595378Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:119)
> 2020-11-19T10:34:23.5596001Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:103)
> 2020-11-19T10:34:23.5596610Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:77)
> 2020-11-19T10:34:23.5597218Z at
> org.apache.flink.table.planner.sinks.SelectTableSinkBase$RowIteratorWrapper.hasNext(SelectTableSinkBase.java:115)
> 2020-11-19T10:34:23.5597811Z at
> org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:355)
> 2020-11-19T10:34:23.5598555Z at
> org.apache.flink.connectors.hive.HiveTableSourceITCase.fetchRows(HiveTableSourceITCase.java:653)
> 2020-11-19T10:34:23.5599407Z at
> org.apache.flink.connectors.hive.HiveTableSourceITCase.testStreamPartitionReadByCreateTime(HiveTableSourceITCase.java:594)
> 2020-11-19T10:34:23.5599982Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-11-19T10:34:23.5600393Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-11-19T10:34:23.5600865Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-11-19T10:34:23.5601300Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-11-19T10:34:23.5601713Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2020-11-19T10:34:23.5602211Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2020-11-19T10:34:23.5602688Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2020-11-19T10:34:23.5603181Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2020-11-19T10:34:23.5603753Z at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> 2020-11-19T10:34:23.5604308Z at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> 2020-11-19T10:34:23.5604780Z at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 2020-11-19T10:34:23.5605114Z at java.lang.Thread.run(Thread.java:748)
> 2020-11-19T10:34:23.5605299Z
> 2020-11-19T10:34:24.4180149Z [INFO] Running
> org.apache.flink.connectors.hive.TableEnvHiveConnectorITCase
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #15603: [FLINK-22169][sql-client] Beautify the CliTableauResultView when print
flinkbot commented on pull request #15603: URL: https://github.com/apache/flink/pull/15603#issuecomment-819207681 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a230294e88fd9cef432d1d53d41a163318efcf5d (Wed Apr 14 03:47:34 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15604: [FLINK-22265][javadocs]Abnormal document display
flinkbot commented on pull request #15604: URL: https://github.com/apache/flink/pull/15604#issuecomment-819207671 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8ca0e185d9016d9a0b4debafb90b1f59d8bdcee4 (Wed Apr 14 03:47:31 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-22265).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15605: [FLINK-21996][coordination] - Part 3&4: Ensure OperatorEvent transport losses are handled
flinkbot commented on pull request #15605: URL: https://github.com/apache/flink/pull/15605#issuecomment-819207661 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit d2111f6d5a17e9263399aac7c188083c793b418d (Wed Apr 14 03:47:29 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-22270) Python test pipeline no output for 900 seconds
Guowei Ma created FLINK-22270: - Summary: Python test pipeline no output for 900 seconds Key: FLINK-22270 URL: https://issues.apache.org/jira/browse/FLINK-22270 Project: Flink Issue Type: Bug Components: API / Python Affects Versions: 1.12.2 Reporter: Guowei Ma https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16481=logs=9cada3cb-c1d3-5621-16da-0f718fb86602=455fddbf-5921-5b71-25ac-92992ad80b28=18771 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22270) Python test pipeline no output for 900 seconds
[ https://issues.apache.org/jira/browse/FLINK-22270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guowei Ma updated FLINK-22270: -- Affects Version/s: (was: 1.12.2) 1.11.3 > Python test pipeline no output for 900 seconds > -- > > Key: FLINK-22270 > URL: https://issues.apache.org/jira/browse/FLINK-22270 > Project: Flink > Issue Type: Bug > Components: API / Python >Affects Versions: 1.11.3 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16481=logs=9cada3cb-c1d3-5621-16da-0f718fb86602=455fddbf-5921-5b71-25ac-92992ad80b28=18771 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15602: [FLINK-22264][docs] Fix misleading statement about Flink Job Cluster Kubernetes Support in Flink Architecture page
flinkbot edited a comment on pull request #15602: URL: https://github.com/apache/flink/pull/15602#issuecomment-819181043 ## CI report: * d6c10842849e1d47bb035ac4ee140395f9214f3b UNKNOWN * a63a44d02327232df329255b5e5ab40484238b92 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16493) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15600: [FLINK-22144][runtime][runtime-web] Adds more documentation and moves option
flinkbot edited a comment on pull request #15600: URL: https://github.com/apache/flink/pull/15600#issuecomment-819054138 ## CI report: * c104e20035af2d1fcecd864d7d906b8d3bd07da9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16483) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15578: [FLINK-21431][upsert-kafka] Use testcontainers for KafkaTable ITCase
flinkbot edited a comment on pull request #15578: URL: https://github.com/apache/flink/pull/15578#issuecomment-817840111 ## CI report: * 0bde4a74b86b0c4dbb1537cda28760bfe2265238 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16468) * beff13ab2dabb61df1ee4017853c758487fa7063 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16496) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fsk119 closed pull request #15562: [FLINK-22169][sql-client] Beautify the CliTableauResultView when print
fsk119 closed pull request #15562: URL: https://github.com/apache/flink/pull/15562 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] KurtYoung commented on pull request #15562: [FLINK-22169][sql-client] Beautify the CliTableauResultView when print
KurtYoung commented on pull request #15562: URL: https://github.com/apache/flink/pull/15562#issuecomment-819205886 you can close this one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-17510) StreamingKafkaITCase. testKafka timeouts on downloading Kafka
[
https://issues.apache.org/jira/browse/FLINK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320674#comment-17320674
]
Guowei Ma commented on FLINK-17510:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16481=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=2b7514ee-e706-5046-657b-3430666e7bd9=17314
> StreamingKafkaITCase. testKafka timeouts on downloading Kafka
> -
>
> Key: FLINK-17510
> URL: https://issues.apache.org/jira/browse/FLINK-17510
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Connectors / Kafka, Tests
>Affects Versions: 1.11.3, 1.12.1, 1.13.0
>Reporter: Robert Metzger
>Priority: Critical
> Labels: test-stability
>
> CI:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=585=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=1e2bbe5b-4657-50be-1f07-d84bfce5b1f5
> {code}
> 2020-05-05T00:06:49.7268716Z [INFO]
> ---
> 2020-05-05T00:06:49.7268938Z [INFO] T E S T S
> 2020-05-05T00:06:49.7269282Z [INFO]
> ---
> 2020-05-05T00:06:50.5336315Z [INFO] Running
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8603439Z [ERROR] Tests run: 3, Failures: 0, Errors: 2,
> Skipped: 0, Time elapsed: 276.323 s <<< FAILURE! - in
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8604882Z [ERROR] testKafka[1:
> kafka-version:0.11.0.2](org.apache.flink.tests.util.kafka.StreamingKafkaITCase)
> Time elapsed: 120.024 s <<< ERROR!
> 2020-05-05T00:11:26.8605942Z java.io.IOException: Process ([wget, -q, -P,
> /tmp/junit2815750531595874769/downloads/1290570732,
> https://archive.apache.org/dist/kafka/0.11.0.2/kafka_2.11-0.11.0.2.tgz])
> exceeded timeout (12) or number of retries (3).
> 2020-05-05T00:11:26.8606732Z at
> org.apache.flink.tests.util.AutoClosableProcess$AutoClosableProcessBuilder.runBlockingWithRetry(AutoClosableProcess.java:132)
> 2020-05-05T00:11:26.8607321Z at
> org.apache.flink.tests.util.cache.AbstractDownloadCache.getOrDownload(AbstractDownloadCache.java:127)
> 2020-05-05T00:11:26.8607826Z at
> org.apache.flink.tests.util.cache.LolCache.getOrDownload(LolCache.java:31)
> 2020-05-05T00:11:26.8608343Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.setupKafkaDist(LocalStandaloneKafkaResource.java:98)
> 2020-05-05T00:11:26.8608892Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.before(LocalStandaloneKafkaResource.java:92)
> 2020-05-05T00:11:26.8609602Z at
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:46)
> 2020-05-05T00:11:26.8610026Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2020-05-05T00:11:26.8610553Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2020-05-05T00:11:26.8610958Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2020-05-05T00:11:26.8611388Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2020-05-05T00:11:26.8612214Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2020-05-05T00:11:26.8612706Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-05-05T00:11:26.8613109Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-05-05T00:11:26.8613551Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-05-05T00:11:26.8614019Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-05-05T00:11:26.8614442Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2020-05-05T00:11:26.8614869Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-05-05T00:11:26.8615251Z at
> org.junit.runners.Suite.runChild(Suite.java:128)
> 2020-05-05T00:11:26.8615654Z at
> org.junit.runners.Suite.runChild(Suite.java:27)
> 2020-05-05T00:11:26.8616060Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-05-05T00:11:26.8616465Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-05-05T00:11:26.8616893Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-05-05T00:11:26.8617893Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-05-05T00:11:26.8618490Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2020-05-05T00:11:26.8619056Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-05-05T00:11:26.8619589Z at
> org.junit.runners.Suite.runChild(Suite.java:128)
> 2020-05-05T00:11:26.8620073Z at
> org.junit.runners.Suite.runChild(Suite.java:27)
>
[GitHub] [flink] StephanEwen opened a new pull request #15605: [FLINK-21996][coordination] - Part 3&4: Ensure OperatorEvent transport losses are handled
StephanEwen opened a new pull request #15605: URL: https://github.com/apache/flink/pull/15605 ## This is based on #15557 and #15601 Only the last commit, named *"[FLINK-21996][coordination] Ensure exactly-once guarantees for OperatorEvent RPCs"* is relevant. It is a fairly small one, compared to the previous PR. ## What is the purpose of the change This is the third and final PR on the way to fixing [FLINK-21996]. The overall steps of the fix are: 1. Add a test and did preliminary refactoring (#15557) 2. Fix [FLINK-18071] to make sure target tasks for OperatorEvents are well-defined (#15601) 3. Failover for target task on timeouts on *Acks* for sending `OperatorEvents` **(This PR)** 4. Delay operator coordinator checkpoint completion until all pending event acks are received or failed. **(This PR)** This PR activates the `OperatorEventSendingCheckpointITCase` which ran successfully 500x in a row with this patch. ## Changes in this PR **Ensure checkpoints don't happen while we have an unclear status of in-flight OperatorCoordinator events.** We cannot complete a checkpoint while there are events that have been sent before a checkpoint and have not yet been either confirmed received or confirmed failed. We wouldn't know whether to assume the data carried by those events is pre-checkpoint (and handled by the receiving tasks) or post checkpoints (and to be handled on the coordinator). This change delays the confirmation of checkpoints until there is clarity about the status of events. Furthermore, if event sending possibly failed, the current checkpoint is discarded. We anyways trigger a recovery of the event's target subtask in that case. **When events are presumed lost in transfer, fail the target task** Because we don't know whether the events were actually lost, or just the ack is lost, we trigger a failover for the receiving task. This both restarts the actual receiver task from the latest checkpoint and notifies the coordinator to reset its view of that task to the latest checkpoint. This is a safe option to restore consistency. In a future optimization, we can add idempotent retries to the event sending (in Flink, which would be on top of what Akka already does), to reduce the number of cases where such a failover is necessary. However, we need to keep the failover as a last resort, because we cannot keep retrying infinitely. ## Verifying this change This is hard to test in a real-world setup, it would need very specific network setup that provokes just the right package losses and very severe thread races. The `CoordinatorEventsExactlyOnceITCase` and `OperatorEventSendingCheckpointITCase` target the relevant scenarios pretty precisely. Both fail reliably on `master` and succeed with this series of patches. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22122) StreamingKafkaITCase Fail due to TestTimedOutException
[
https://issues.apache.org/jira/browse/FLINK-22122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320673#comment-17320673
]
Guowei Ma commented on FLINK-22122:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16447=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=27109
> StreamingKafkaITCase Fail due to TestTimedOutException
> --
>
> Key: FLINK-22122
> URL: https://issues.apache.org/jira/browse/FLINK-22122
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Major
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16059=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=27391
> {code:java}
> Apr 06 08:34:01 org.junit.runners.model.TestTimedOutException: test timed out
> after 3 minutes
> Apr 06 08:34:01 at java.lang.Object.wait(Native Method)
> Apr 06 08:34:01 at java.lang.Object.wait(Object.java:502)
> Apr 06 08:34:01 at java.lang.UNIXProcess.waitFor(UNIXProcess.java:395)
> Apr 06 08:34:01 at
> org.apache.flink.tests.util.flink.FlinkDistribution.submitJob(FlinkDistribution.java:194)
> Apr 06 08:34:01 at
> org.apache.flink.tests.util.flink.LocalStandaloneFlinkResource$StandaloneClusterController.submitJob(LocalStandaloneFlinkResource.java:200)
> Apr 06 08:34:01 at
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase.testKafka(StreamingKafkaITCase.java:109)
> Apr 06 08:34:01 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Apr 06 08:34:01 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Apr 06 08:34:01 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Apr 06 08:34:01 at java.lang.reflect.Method.invoke(Method.java:498)
> Apr 06 08:34:01 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Apr 06 08:34:01 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Apr 06 08:34:01 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Apr 06 08:34:01 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Apr 06 08:34:01 at
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 06 08:34:01 at
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 06 08:34:01 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> Apr 06 08:34:01 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> Apr 06 08:34:01 at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Apr 06 08:34:01 at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-19863) SQLClientHBaseITCase.testHBase failed with "java.io.IOException: Process failed due to timeout"
[
https://issues.apache.org/jira/browse/FLINK-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320672#comment-17320672
]
Guowei Ma commented on FLINK-19863:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16450=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=27710
> SQLClientHBaseITCase.testHBase failed with "java.io.IOException: Process
> failed due to timeout"
> ---
>
> Key: FLINK-19863
> URL: https://issues.apache.org/jira/browse/FLINK-19863
> Project: Flink
> Issue Type: Bug
> Components: Connectors / HBase
>Affects Versions: 1.12.0, 1.12.3
>Reporter: Dian Fu
>Assignee: Leonard Xu
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.12.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8541=logs=91bf6583-3fb2-592f-e4d4-d79d79c3230a=3425d8ba-5f03-540a-c64b-51b8481bf7d6
> {code}
> 00:50:02,589 [main] INFO
> org.apache.flink.tests.util.flink.FlinkDistribution [] - Stopping
> Flink cluster.
> 00:50:04,106 [main] INFO
> org.apache.flink.tests.util.flink.FlinkDistribution [] - Stopping
> Flink cluster.
> 00:50:04,741 [main] INFO
> org.apache.flink.tests.util.flink.LocalStandaloneFlinkResource [] - Backed up
> logs to
> /home/vsts/work/1/s/flink-end-to-end-tests/artifacts/flink-b3924665-1ac9-4309-8255-20f0dc94e7b9.
> 00:50:04,788 [main] INFO
> org.apache.flink.tests.util.hbase.LocalStandaloneHBaseResource [] - Stopping
> HBase Cluster
> 00:50:16,243 [main] ERROR
> org.apache.flink.tests.util.hbase.SQLClientHBaseITCase [] -
>
> Test testHBase[0:
> hbase-version:1.4.3](org.apache.flink.tests.util.hbase.SQLClientHBaseITCase)
> failed with:
> java.io.IOException: Process failed due to timeout.
> at
> org.apache.flink.tests.util.AutoClosableProcess$AutoClosableProcessBuilder.runBlocking(AutoClosableProcess.java:130)
> at
> org.apache.flink.tests.util.AutoClosableProcess$AutoClosableProcessBuilder.runBlocking(AutoClosableProcess.java:108)
> at
> org.apache.flink.tests.util.flink.FlinkDistribution.submitSQLJob(FlinkDistribution.java:221)
> at
> org.apache.flink.tests.util.flink.LocalStandaloneFlinkResource$StandaloneClusterController.submitSQLJob(LocalStandaloneFlinkResource.java:196)
> at
> org.apache.flink.tests.util.hbase.SQLClientHBaseITCase.executeSqlStatements(SQLClientHBaseITCase.java:215)
> at
> org.apache.flink.tests.util.hbase.SQLClientHBaseITCase.testHBase(SQLClientHBaseITCase.java:152)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-22265) Abnormal document display
[ https://issues.apache.org/jira/browse/FLINK-22265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-22265: --- Labels: pull-request-available (was: ) > Abnormal document display > - > > Key: FLINK-22265 > URL: https://issues.apache.org/jira/browse/FLINK-22265 > Project: Flink > Issue Type: Improvement > Components: API / Core >Reporter: BoYi Zhang >Priority: Minor > Labels: pull-request-available > Attachments: image-2021-04-14-11-00-24-999.png, > image-2021-04-14-11-01-47-226.png > > > > As shown in the figure, the document displays an exception : > > !image-2021-04-14-11-00-24-999.png! > > After repair : > > !image-2021-04-14-11-01-47-226.png! > > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] BoYiZhang opened a new pull request #15604: [FLINK-22265][javadocs]Abnormal document display
BoYiZhang opened a new pull request #15604: URL: https://github.com/apache/flink/pull/15604 fix Abnormal document display -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22248) JobMasterStopWithSavepointITCase.suspendWithSavepointWithoutComplicationsShouldSucceedAndLeadJobToFinished fail because of TestTimedOutException
[
https://issues.apache.org/jira/browse/FLINK-22248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320668#comment-17320668
]
Guowei Ma commented on FLINK-22248:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=a57e0635-3fad-5b08-57c7-a4142d7d6fa9=5360d54c-8d94-5d85-304e-a89267eb785a=9770
> JobMasterStopWithSavepointITCase.suspendWithSavepointWithoutComplicationsShouldSucceedAndLeadJobToFinished
> fail because of TestTimedOutException
>
>
> Key: FLINK-22248
> URL: https://issues.apache.org/jira/browse/FLINK-22248
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Assignee: Dawid Wysakowicz
>Priority: Critical
> Labels: test-stability
> Fix For: 1.13.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16405=logs=39d5b1d5-3b41-54dc-6458-1e2ddd1cdcf3=a99e99c7-21cd-5a1f-7274-585e62b72f56=4325
> {code:java}
> org.junit.runners.model.TestTimedOutException: test timed out after 5000
> milliseconds
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.setUpJobGraph(JobMasterStopWithSavepointITCase.java:308)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.stopWithSavepointNormalExecutionHelper(JobMasterStopWithSavepointITCase.java:113)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.suspendWithSavepointWithoutComplicationsShouldSucceedAndLeadJobToFinished(JobMasterStopWithSavepointITCase.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] fsk119 commented on pull request #15562: [FLINK-22169][sql-client] Beautify the CliTableauResultView when print
fsk119 commented on pull request #15562: URL: https://github.com/apache/flink/pull/15562#issuecomment-819202821 Continue in #15603. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fsk119 opened a new pull request #15603: [FLINK-22169][sql-client] Beautify the CliTableauResultView when print
fsk119 opened a new pull request #15603: URL: https://github.com/apache/flink/pull/15603 ## What is the purpose of the change *Use `PrintUtils#printAsTableauForm` to calculate column width in batch mode..* ## Verifying this change This change is already covered by existing tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented**) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-22269) JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithRestartsShouldSimplyRestartInSuspend failed.
Guowei Ma created FLINK-22269:
-
Summary:
JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithRestartsShouldSimplyRestartInSuspend
failed.
Key: FLINK-22269
URL: https://issues.apache.org/jira/browse/FLINK-22269
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing, Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Guowei Ma
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=a57e0635-3fad-5b08-57c7-a4142d7d6fa9=5360d54c-8d94-5d85-304e-a89267eb785a=9197
{code:java}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithRestartsHelper(JobMasterStopWithSavepointITCase.java:195)
at
org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithRestartsShouldSimplyRestartInSuspend(JobMasterStopWithSavepointITCase.java:161)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15538: [FLINK-21032][table] Not use ParallelFiniteTestSource in CompactionITCaseBase
flinkbot edited a comment on pull request #15538: URL: https://github.com/apache/flink/pull/15538#issuecomment-816475885 ## CI report: * 46ce52e1b6dde0b57aefe08d525ccfdb89500aa7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16263) * d5580c1c4dfd9b257223c8906737b9675188ed44 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16495) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15525: [FLINK-22154][table-planner-blink] Fix bug where PushFilterIntoTableSourceScanRule fails to deal with IN expressions
flinkbot edited a comment on pull request #15525: URL: https://github.com/apache/flink/pull/15525#issuecomment-815836876 ## CI report: * 8179c44907265a7a7aa4294888c003a3a67d5f7b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16494) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 84805e9cf0d86999e0c839be446f87637dc09de8 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16475) * 0073a4beb7ecc117dbd16960bfd3aa17e948dd79 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16491) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22266) JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate fail
[
https://issues.apache.org/jira/browse/FLINK-22266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guowei Ma updated FLINK-22266:
--
Summary:
JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate
fail (was:
JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper
fail)
> JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate
> fail
> -
>
> Key: FLINK-22266
> URL: https://issues.apache.org/jira/browse/FLINK-22266
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0=3884
> {code:java}
> [ERROR]
> throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase)
> Time elapsed: 0.154 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper(JobMasterStopWithSavepointITCase.java:154)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(JobMasterStopWithSavepointITCase.java:138)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] JingsongLi commented on pull request #15533: [FLINK-22063][table] Fix bug that some join conditions are lost when …
JingsongLi commented on pull request #15533: URL: https://github.com/apache/flink/pull/15533#issuecomment-819200390 @cshuo Please add some comments to explain why -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22266) JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper fail
[
https://issues.apache.org/jira/browse/FLINK-22266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320666#comment-17320666
]
Guowei Ma commented on FLINK-22266:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=a57e0635-3fad-5b08-57c7-a4142d7d6fa9=5360d54c-8d94-5d85-304e-a89267eb785a=9177
> JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper
> fail
> --
>
> Key: FLINK-22266
> URL: https://issues.apache.org/jira/browse/FLINK-22266
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0=3884
> {code:java}
> [ERROR]
> throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase)
> Time elapsed: 0.154 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper(JobMasterStopWithSavepointITCase.java:154)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(JobMasterStopWithSavepointITCase.java:138)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-22266) JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper fail
[
https://issues.apache.org/jira/browse/FLINK-22266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guowei Ma updated FLINK-22266:
--
Priority: Critical (was: Major)
> JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper
> fail
> --
>
> Key: FLINK-22266
> URL: https://issues.apache.org/jira/browse/FLINK-22266
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0=3884
> {code:java}
> [ERROR]
> throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase)
> Time elapsed: 0.154 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper(JobMasterStopWithSavepointITCase.java:154)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(JobMasterStopWithSavepointITCase.java:138)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-22268) JobMasterStopWithSavepointITCase.testRestartCheckpointCoordinatorIfStopWithSavepointFails fail because of "Not all required tasks are currently running."
Guowei Ma created FLINK-22268: - Summary: JobMasterStopWithSavepointITCase.testRestartCheckpointCoordinatorIfStopWithSavepointFails fail because of "Not all required tasks are currently running." Key: FLINK-22268 URL: https://issues.apache.org/jira/browse/FLINK-22268 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing, Runtime / Coordination Affects Versions: 1.13.0 Reporter: Guowei Ma https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0=3899 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22267) Savepoint an application for source of upsert-kafka, then restart the application from the savepoint, state not be recovered.
Carl created FLINK-22267: Summary: Savepoint an application for source of upsert-kafka, then restart the application from the savepoint, state not be recovered. Key: FLINK-22267 URL: https://issues.apache.org/jira/browse/FLINK-22267 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.12.2 Reporter: Carl Attachments: image-2021-04-14-11-17-00-207.png My operation is as follows： 1. Savepoint an application for source of upsert-kafka 2. Delete the upsert Kafka topic data 3. restart the application from the savepoint 4. Log shows that the offset has been restored, but the state has not been restored What I want to confirm is that restart from savepoint in the source upsert-kafka application not restore the state from savepoint state but restore the state from earliest offset kafka message? If so, why reset offset： !image-2021-04-14-11-17-00-207.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22266) JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper fail
Guowei Ma created FLINK-22266:
-
Summary:
JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper
fail
Key: FLINK-22266
URL: https://issues.apache.org/jira/browse/FLINK-22266
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing, Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Guowei Ma
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0=3884
{code:java}
[ERROR]
throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase)
Time elapsed: 0.154 s <<< FAILURE!
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper(JobMasterStopWithSavepointITCase.java:154)
at
org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(JobMasterStopWithSavepointITCase.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-22265) Abnormal document display
[ https://issues.apache.org/jira/browse/FLINK-22265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320664#comment-17320664 ] BoYi Zhang commented on FLINK-22265: I'll fix that > Abnormal document display > - > > Key: FLINK-22265 > URL: https://issues.apache.org/jira/browse/FLINK-22265 > Project: Flink > Issue Type: Improvement > Components: API / Core >Reporter: BoYi Zhang >Priority: Minor > Attachments: image-2021-04-14-11-00-24-999.png, > image-2021-04-14-11-01-47-226.png > > > > As shown in the figure, the document displays an exception : > > !image-2021-04-14-11-00-24-999.png! > > After repair : > > !image-2021-04-14-11-01-47-226.png! > > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22262) Flink on Kubernetes ConfigMaps are created without OwnerReference
[
https://issues.apache.org/jira/browse/FLINK-22262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320663#comment-17320663
]
Yang Wang commented on FLINK-22262:
---
I think we could not set the owner reference for the HA related ConfigMaps.
Because it could happen that we delete the K8s resources but want to recover
the Flink jobs. Actually, if you stop your Flink application with cancel, then
I believe the HA related ConfigMaps will be deleted automatically.
Refer to here[1] for more information about the HA related ConfigMaps clean up.
1.
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/ha/kubernetes_ha/#high-availability-data-clean-up
> Flink on Kubernetes ConfigMaps are created without OwnerReference
> -
>
> Key: FLINK-22262
> URL: https://issues.apache.org/jira/browse/FLINK-22262
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Andrea Peruffo
>Priority: Major
>
> According to the documentation:
> [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#manual-resource-cleanup]
> The ConfigMaps created along with the Flink deployment is supposed to have an
> OwnerReference pointing to the Deployment itself, unfortunately, this doesn't
> happen and causes all sorts of issues when the classpath and the jars of the
> job are updated.
> i.e.:
> Without manually removing the ConfigMap of the Job I cannot update the Jars
> of the Job.
> Can you please give guidance if there are additional caveats on manually
> removing the ConfigMap? Any other workaround that can be used?
> Thanks in advance.
> Example ConfigMap:
> {{apiVersion: v1}}
> {{data:}}
> {{ address: akka.tcp://flink@10.0.2.13:6123/user/rpc/jobmanager_2}}
> {{ checkpointID-049:
> rO0ABXNyADtvcmcuYXBhY2hlLmZsaW5rLnJ1bnRpbWUuc3RhdGUuUmV0cmlldmFibGVTdHJlYW1TdGF0ZUhhbmRsZQABHhjxVZcrAgABTAAYd3JhcHBlZFN0cmVhbVN0YXRlSGFuZGxldAAyTG9yZy9hcGFjaGUvZmxpbmsvcnVudGltZS9zdGF0ZS9TdHJlYW1TdGF0ZUhhbmRsZTt4cHNyADlvcmcuYXBhY2hlLmZsaW5rLnJ1bnRpbWUuc3RhdGUuZmlsZXN5c3RlbS5GaWxlU3RhdGVIYW5kbGUE3HXYYr0bswIAAkoACXN0YXRlU2l6ZUwACGZpbGVQYXRodAAfTG9yZy9hcGFjaGUvZmxpbmsvY29yZS9mcy9QYXRoO3hwAAABOEtzcgAdb3JnLmFwYWNoZS5mbGluay5jb3JlLmZzLlBhdGgAAQIAAUwAA3VyaXQADkxqYXZhL25ldC9VUkk7eHBzcgAMamF2YS5uZXQuVVJJrAF4LkOeSasDAAFMAAZzdHJpbmd0ABJMamF2YS9sYW5nL1N0cmluZzt4cHQAUC9tbnQvZmxpbmsvc3RvcmFnZS9rc2hhL3RheGktcmlkZS1mYXJlLXByb2Nlc3Nvci9jb21wbGV0ZWRDaGVja3BvaW50MDQ0YTc2OWRkNDgxeA==}}
> {{ counter: "50"}}
> {{ sessionId: 0c2b69ee-6b41-48d3-b7fd-1bf2eda94f0f}}
> {{kind: ConfigMap}}
> {{metadata:}}
> {{ annotations:}}
> {{ control-plane.alpha.kubernetes.io/leader:
> '\{"holderIdentity":"0f25a2cc-e212-46b0-8ba9-faac0732a316","leaseDuration":15.0,"acquireTime":"2021-04-13T14:30:51.439000Z","renewTime":"2021-04-13T14:39:32.011000Z","leaderTransitions":105}'}}
> {{ creationTimestamp: "2021-04-13T14:30:51Z"}}
> {{ labels:}}
> {{ app: taxi-ride-fare-processor}}
> {{ configmap-type: high-availability}}
> {{ type: flink-native-kubernetes}}
> {{ name:
> taxi-ride-fare-processor--jobmanager-leader}}
> {{ namespace: taxi-ride-fare}}
> {{ resourceVersion: "64100"}}
> {{ selfLink:
> /api/v1/namespaces/taxi-ride-fare/configmaps/taxi-ride-fare-processor--jobmanager-leader}}
> {{ uid: 9f912495-382a-45de-a789-fd5ad2a2459d}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15578: [FLINK-21431][upsert-kafka] Use testcontainers for KafkaTable ITCase
flinkbot edited a comment on pull request #15578: URL: https://github.com/apache/flink/pull/15578#issuecomment-817840111 ## CI report: * 0bde4a74b86b0c4dbb1537cda28760bfe2265238 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16468) * beff13ab2dabb61df1ee4017853c758487fa7063 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15602: [FLINK-22264][docs] Fix misleading statement about Flink Job Cluster Kubernetes Support in Flink Architecture page
flinkbot edited a comment on pull request #15602: URL: https://github.com/apache/flink/pull/15602#issuecomment-819181043 ## CI report: * d6c10842849e1d47bb035ac4ee140395f9214f3b UNKNOWN * a63a44d02327232df329255b5e5ab40484238b92 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15538: [FLINK-21032][table] Not use ParallelFiniteTestSource in CompactionITCaseBase
flinkbot edited a comment on pull request #15538: URL: https://github.com/apache/flink/pull/15538#issuecomment-816475885 ## CI report: * 46ce52e1b6dde0b57aefe08d525ccfdb89500aa7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16263) * d5580c1c4dfd9b257223c8906737b9675188ed44 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15525: [FLINK-22154][table-planner-blink] Fix bug where PushFilterIntoTableSourceScanRule fails to deal with IN expressions
flinkbot edited a comment on pull request #15525: URL: https://github.com/apache/flink/pull/15525#issuecomment-815836876 ## CI report: * 7f423888b031f3ebbf56596984709158de3a47d1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16460) * 8179c44907265a7a7aa4294888c003a3a67d5f7b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 84805e9cf0d86999e0c839be446f87637dc09de8 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16475) * 0073a4beb7ecc117dbd16960bfd3aa17e948dd79 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-22265) Abnormal document display
BoYi Zhang created FLINK-22265: -- Summary: Abnormal document display Key: FLINK-22265 URL: https://issues.apache.org/jira/browse/FLINK-22265 Project: Flink Issue Type: Improvement Components: API / Core Reporter: BoYi Zhang Attachments: image-2021-04-14-11-00-24-999.png, image-2021-04-14-11-01-47-226.png As shown in the figure, the document displays an exception : !image-2021-04-14-11-00-24-999.png! After repair : !image-2021-04-14-11-01-47-226.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22264) Fix misleading statement about per-job mode support for Kubernetes in Concept/Flink Architecture page
[ https://issues.apache.org/jira/browse/FLINK-22264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320660#comment-17320660 ] Fuyao Li commented on FLINK-22264: -- Thanks. Please refer to the PR I submitted for details. https://github.com/apache/flink/pull/15602 > Fix misleading statement about per-job mode support for Kubernetes in > Concept/Flink Architecture page > - > > Key: FLINK-22264 > URL: https://issues.apache.org/jira/browse/FLINK-22264 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.12.2 >Reporter: Fuyao Li >Assignee: Fuyao Li >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > I noticed a conflict in the document for per-job mode support for Kubernetes. > In the doc here [1], it mentions > in a Flink Job Cluster, the available cluster manager (like YARN or > Kubernetes) is used to spin up a cluster for each submitted job and this > cluster is available to that job only. > It implies per job mode is supported in Kubernetes. > > However, in the docs [2] and [3], it clearly points out per-job mode is not > supported in Kubernetes. > > To avoid the misunderstanding, I think we should fix the statement in the > concept page. I had a discussion with Yang Wang on flink user mailing list > earlier. (link still not available in the archive for now.) > > [1] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/concepts/flink-architecture/#flink-job-cluster] > [2] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#per-job-cluster-mode] > [3] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22264) Fix misleading statement about per-job mode support for Kubernetes in Concept/Flink Architecture page
[ https://issues.apache.org/jira/browse/FLINK-22264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320659#comment-17320659 ] Yang Wang commented on FLINK-22264: --- I have assigned this ticket to you. > Fix misleading statement about per-job mode support for Kubernetes in > Concept/Flink Architecture page > - > > Key: FLINK-22264 > URL: https://issues.apache.org/jira/browse/FLINK-22264 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.12.2 >Reporter: Fuyao Li >Assignee: Fuyao Li >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > I noticed a conflict in the document for per-job mode support for Kubernetes. > In the doc here [1], it mentions > in a Flink Job Cluster, the available cluster manager (like YARN or > Kubernetes) is used to spin up a cluster for each submitted job and this > cluster is available to that job only. > It implies per job mode is supported in Kubernetes. > > However, in the docs [2] and [3], it clearly points out per-job mode is not > supported in Kubernetes. > > To avoid the misunderstanding, I think we should fix the statement in the > concept page. I had a discussion with Yang Wang on flink user mailing list > earlier. (link still not available in the archive for now.) > > [1] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/concepts/flink-architecture/#flink-job-cluster] > [2] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#per-job-cluster-mode] > [3] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-22263:
Fix Version/s: 1.13.0
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
> Fix For: 1.13.0
>
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
>
[jira] [Closed] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-22263.
---
Resolution: Fixed
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
>
[jira] [Updated] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kurt Young updated FLINK-22263:
---
Component/s: Table SQL / Planner
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
>
[jira] [Updated] (FLINK-22263) Using TIMESTAMPADD function with partition value has some problem when push partition into TableSource
[
https://issues.apache.org/jira/browse/FLINK-22263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kurt Young updated FLINK-22263:
---
Affects Version/s: 1.12.2
> Using TIMESTAMPADD function with partition value has some problem when push
> partition into TableSource
>
>
> Key: FLINK-22263
> URL: https://issues.apache.org/jira/browse/FLINK-22263
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.12.2
>Reporter: hehuiyuan
>Priority: Major
>
> SQL (table api):
> {code:java}
> CREATE CATALOG myhive
> WITH (
> 'type' = 'hive',
> 'default-database' = 'hhy'
> );
> INSERT INTO default_catalog.default_database.table_sink select * from
> myhive.hhy.tmp_flink_test where dt=CAST(TIMESTAMPADD(DAY, -1, CURRENT_DATE)
> as varchar);
> {code}
>
> Error log:
> {code:java}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Data type 'INTERVAL SECOND(3) NOT NULL' with conversion class
> 'java.time.Duration' does not support a value literal of class
> 'java.math.BigDecimal'.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Data type 'INTERVAL SECOND(3)
> NOT NULL' with conversion class 'java.time.Duration' does not support a value
> literal of class 'java.math.BigDecimal'. at
> org.apache.flink.table.expressions.ValueLiteralExpression.validateValueDataType(ValueLiteralExpression.java:286)
> at
> org.apache.flink.table.expressions.ValueLiteralExpression.(ValueLiteralExpression.java:79)
> at
> org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral(ApiExpressionUtils.java:251)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:432)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitLiteral(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1173) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:439)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:340)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter$$anonfun$7.apply(RexNodeExtractor.scala:440)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at
[GitHub] [flink] flinkbot commented on pull request #15602: [FLINK-22264][docs] Fix misleading statement about Flink Job Cluster Kubernetes Support in Flink Architecture page
flinkbot commented on pull request #15602: URL: https://github.com/apache/flink/pull/15602#issuecomment-819181043 ## CI report: * d6c10842849e1d47bb035ac4ee140395f9214f3b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] JingsongLi commented on pull request #15538: [FLINK-21032][table] Not use ParallelFiniteTestSource in CompactionITCaseBase
JingsongLi commented on pull request #15538: URL: https://github.com/apache/flink/pull/15538#issuecomment-819180941 @leonardBang Nice catch! I have fixed source and compact operator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15523: [FLINK-22109][table-planner-blink] Resolve misleading exception message in invalid nested function
flinkbot edited a comment on pull request #15523: URL: https://github.com/apache/flink/pull/15523#issuecomment-815495525 ## CI report: * 550f2676535859243f738113dd5258edd0655e81 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16266) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16183) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16329) * 0acc67e6b93b5c4a2cc444af555d9f80a9cd0b63 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16489) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-22245) Reuse PrintUtils.MAX_COLUMN_WIDTH in CliChangelogResultView and CliTableResultView
[
https://issues.apache.org/jira/browse/FLINK-22245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-22245.
---
Resolution: Fixed
Fixed in master: a66487d4d2f77bb09b9961655cc5922c3e5c098c
> Reuse PrintUtils.MAX_COLUMN_WIDTH in CliChangelogResultView and
> CliTableResultView
> --
>
> Key: FLINK-22245
> URL: https://issues.apache.org/jira/browse/FLINK-22245
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.13.0
>Reporter: Shengkai Fang
>Assignee: Shengkai Fang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
> Attachments: sql-client.png
>
>
> The max column width in CliChangelogResultView and CliTableResultView is 25.
> In the picture, it's not enough. We should reuse the
> {{PrintUtils.MAX_COLUMN_WIDTH}}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong merged pull request #15580: [FLINK-22245][sql-client] Reuse PrintUtils.MAX_COLUMN_WIDTH in CliCha…
wuchong merged pull request #15580: URL: https://github.com/apache/flink/pull/15580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21992) Fix availability notification in UnionInputGate
[
https://issues.apache.org/jira/browse/FLINK-21992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320653#comment-17320653
]
Guowei Ma commented on FLINK-21992:
---
We also see the same stack in our internal tests without unaligned checkpoint
enabled.
{code:java}
[stat_date, cate_id, user_id, biz, MIN(visit_time_int) FILTER $f5 AS min$0,
MIN(visit_time_int) FILTER $f6 AS min$1, MIN(visit_time_int) FILTER $f7 AS
min$2, MIN(visit_time_int) FILTER $f8 AS min$3, MIN(visit_time_int) FILTER $f9
AS min$4]))) (68/256)#0" Id=101 WAITING on
java.util.concurrent.CompletableFuture$Signaller@2e0b208c
at sun.misc.Unsafe.park(Native Method)
- waiting on java.util.concurrent.CompletableFuture$Signaller@2e0b208c
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:189)
at
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at
java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at
org.apache.flink.runtime.io.network.buffer.LocalBufferPool.requestMemorySegmentBlocking(LocalBufferPool.java:319)
at
org.apache.flink.runtime.io.network.buffer.LocalBufferPool.requestBufferBuilderBlocking(LocalBufferPool.java:291)
...
{code}
> Fix availability notification in UnionInputGate
> ---
>
> Key: FLINK-21992
> URL: https://issues.apache.org/jira/browse/FLINK-21992
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.12.2, 1.13.0
>Reporter: Arvid Heise
>Assignee: Arvid Heise
>Priority: Blocker
> Labels: pull-request-available
>
> A user on mailing list reported that his job gets stuck with unaligned
> checkpoint enabled.
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Source-Operators-Stuck-in-the-requestBufferBuilderBlocking-tt42530.html
> We received two similar reports in the past, but the users didn't follow up,
> so it was not as easy to diagnose as this time where the initial report
> already contains many relevant data points.
> Beside a buffer leak, there could also be an issue with priority notification.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-22264) Fix misleading statement about per-job mode support for Kubernetes in Concept/Flink Architecture page
[ https://issues.apache.org/jira/browse/FLINK-22264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang reassigned FLINK-22264: - Assignee: Fuyao Li > Fix misleading statement about per-job mode support for Kubernetes in > Concept/Flink Architecture page > - > > Key: FLINK-22264 > URL: https://issues.apache.org/jira/browse/FLINK-22264 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.12.2 >Reporter: Fuyao Li >Assignee: Fuyao Li >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > I noticed a conflict in the document for per-job mode support for Kubernetes. > In the doc here [1], it mentions > in a Flink Job Cluster, the available cluster manager (like YARN or > Kubernetes) is used to spin up a cluster for each submitted job and this > cluster is available to that job only. > It implies per job mode is supported in Kubernetes. > > However, in the docs [2] and [3], it clearly points out per-job mode is not > supported in Kubernetes. > > To avoid the misunderstanding, I think we should fix the statement in the > concept page. I had a discussion with Yang Wang on flink user mailing list > earlier. (link still not available in the archive for now.) > > [1] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/concepts/flink-architecture/#flink-job-cluster] > [2] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#per-job-cluster-mode] > [3] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] leonardBang commented on pull request #15533: [FLINK-22063][table] Fix bug that some join conditions are lost when …
leonardBang commented on pull request #15533: URL: https://github.com/apache/flink/pull/15533#issuecomment-819176809 The failed test is not related to this PR, cc @JingsongLi could you help merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] xintongsong commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
xintongsong commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-819175527 Thanks for preparing this PR, @galenwarren. I'll try to take a look asap. Quick response to your questions. - The licensing issues are described [here](https://cwiki.apache.org/confluence/display/FLINK/Licensing). In short, you need to manually create the NOTICE file. The script is mainly used for generating the root NOTICE file from those of the sub-modules. - I think `commit` should fail if the file is already committed. The contract of this interface says it publishes the file, making it visible. We should not allow a job restarted from an earlier checkpoint/savepoint to overwrite a published file. BTW, AZP failed during compiling. Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-21012) AvroFileFormatFactory uses non-deserializable lambda function
[
https://issues.apache.org/jira/browse/FLINK-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee closed FLINK-21012.
Resolution: Fixed
master (1.13): 7cb5d1c1977a0ac65e0954ea0abf49461a4c0e6c
1.12: 0073a4beb7ecc117dbd16960bfd3aa17e948dd79
> AvroFileFormatFactory uses non-deserializable lambda function
> -
>
> Key: FLINK-21012
> URL: https://issues.apache.org/jira/browse/FLINK-21012
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.12.0, 1.12.1
>Reporter: Ingo Bürk
>Assignee: Ingo Bürk
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.13.0, 1.12.3
>
>
> In AvroFileFormatFactory#RowDataAvroWriterFactory a lambda function is used
> to create the factory. This can causes
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Invalid lambda
> deserialization{code}
> There's other similar issues like FLINK-20147, FLINK-18857 and FLINK-18006
> and the solution so far seems to have been to replace the lambda with an
> anonymous class instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-21012) AvroFileFormatFactory uses non-deserializable lambda function
[
https://issues.apache.org/jira/browse/FLINK-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320644#comment-17320644
]
Jingsong Lee edited comment on FLINK-21012 at 4/14/21, 2:22 AM:
[~airblader] Yes~ Then we can make sure the test passes and we can merge it.
was (Author: lzljs3620320):
[~airblader] Yes~
> AvroFileFormatFactory uses non-deserializable lambda function
> -
>
> Key: FLINK-21012
> URL: https://issues.apache.org/jira/browse/FLINK-21012
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.12.0, 1.12.1
>Reporter: Ingo Bürk
>Assignee: Ingo Bürk
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.13.0, 1.12.3
>
>
> In AvroFileFormatFactory#RowDataAvroWriterFactory a lambda function is used
> to create the factory. This can causes
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Invalid lambda
> deserialization{code}
> There's other similar issues like FLINK-20147, FLINK-18857 and FLINK-18006
> and the solution so far seems to have been to replace the lambda with an
> anonymous class instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] JingsongLi merged pull request #15596: [FLINK-21012][format-avro] Replace lambda with anonymous class to prevent serialization issue
JingsongLi merged pull request #15596: URL: https://github.com/apache/flink/pull/15596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21012) AvroFileFormatFactory uses non-deserializable lambda function
[
https://issues.apache.org/jira/browse/FLINK-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320644#comment-17320644
]
Jingsong Lee commented on FLINK-21012:
--
[~airblader] Yes~
> AvroFileFormatFactory uses non-deserializable lambda function
> -
>
> Key: FLINK-21012
> URL: https://issues.apache.org/jira/browse/FLINK-21012
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.12.0, 1.12.1
>Reporter: Ingo Bürk
>Assignee: Ingo Bürk
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.13.0, 1.12.3
>
>
> In AvroFileFormatFactory#RowDataAvroWriterFactory a lambda function is used
> to create the factory. This can causes
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Invalid lambda
> deserialization{code}
> There's other similar issues like FLINK-20147, FLINK-18857 and FLINK-18006
> and the solution so far seems to have been to replace the lambda with an
> anonymous class instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong commented on pull request #15534: [FLINK-22104][sql-client] Fix unstable SqlClientTest.testExecuteSqlFile
wuchong commented on pull request #15534: URL: https://github.com/apache/flink/pull/15534#issuecomment-819173236 The build hangs on sql client tests. Please have a check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15602: [FLINK-22264][docs] Fix misleading statement about Flink Job Cluster Kubernetes Support in Flink Architecture page
flinkbot commented on pull request #15602: URL: https://github.com/apache/flink/pull/15602#issuecomment-819170222 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit d6c10842849e1d47bb035ac4ee140395f9214f3b (Wed Apr 14 02:11:05 UTC 2021) **Warnings:** * Documentation files were touched, but no `docs/content.zh/` files: Update Chinese documentation or file Jira ticket. * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-22264).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22264) Fix misleading statement about per-job mode support for Kubernetes in Concept/Flink Architecture page
[ https://issues.apache.org/jira/browse/FLINK-22264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-22264: --- Labels: pull-request-available (was: ) > Fix misleading statement about per-job mode support for Kubernetes in > Concept/Flink Architecture page > - > > Key: FLINK-22264 > URL: https://issues.apache.org/jira/browse/FLINK-22264 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.12.2 >Reporter: Fuyao Li >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > I noticed a conflict in the document for per-job mode support for Kubernetes. > In the doc here [1], it mentions > in a Flink Job Cluster, the available cluster manager (like YARN or > Kubernetes) is used to spin up a cluster for each submitted job and this > cluster is available to that job only. > It implies per job mode is supported in Kubernetes. > > However, in the docs [2] and [3], it clearly points out per-job mode is not > supported in Kubernetes. > > To avoid the misunderstanding, I think we should fix the statement in the > concept page. I had a discussion with Yang Wang on flink user mailing list > earlier. (link still not available in the archive for now.) > > [1] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/concepts/flink-architecture/#flink-job-cluster] > [2] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#per-job-cluster-mode] > [3] > [https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] FuyaoLi2017 opened a new pull request #15602: [FLINK-22264][docs] Fix misleading statement about Flink Job Cluster Kubernetes Support in Flink Architecture page
FuyaoLi2017 opened a new pull request #15602: URL: https://github.com/apache/flink/pull/15602 ## What is the purpose of the change I noticed a conflict in the document statement for per-job mode support for Kubernetes. In the doc here [1], it mentions in a Flink Job Cluster, the available cluster manager (like YARN or Kubernetes) is used to spin up a cluster for each submitted job and this cluster is available to that job only. It implies per job mode is supported in Kubernetes. However, in the docs [2] and [3], it clearly points out per-job mode is not supported in Kubernetes. To avoid the misunderstanding, I think we should fix the statement in the Flink Architecture page and add an extra hint note here. I had a discussion with @wangyang0918 on flink user mailing list earlier on this issue and he agrees to patch such a change. [1] https://ci.apache.org/projects/flink/flink-docs-master/docs/concepts/flink-architecture/#flink-job-cluster [2] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#per-job-cluster-mode [3] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/ ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (docs) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15523: [FLINK-22109][table-planner-blink] Resolve misleading exception message in invalid nested function
flinkbot edited a comment on pull request #15523: URL: https://github.com/apache/flink/pull/15523#issuecomment-815495525 ## CI report: * 550f2676535859243f738113dd5258edd0655e81 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16266) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16183) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16329) * 0acc67e6b93b5c4a2cc444af555d9f80a9cd0b63 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-17371) Failed to cast decimal from string or double
[ https://issues.apache.org/jira/browse/FLINK-17371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee closed FLINK-17371. Resolution: Fixed master (1.13): f6fc76936cc3302e4784b334c8e392abe199e992 dd71937152bd4b4ecb70549ff1e62274fd229296 > Failed to cast decimal from string or double > - > > Key: FLINK-17371 > URL: https://issues.apache.org/jira/browse/FLINK-17371 > Project: Flink > Issue Type: Improvement > Components: Table SQL / Runtime >Reporter: Kurt Young >Assignee: Jingsong Lee >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > see: > [https://github.com/apache/flink/blob/master/flink-table/flink-table-planner-blink/src/test/scala/org/apache/flink/table/planner/expressions/DecimalTypeTest.scala#L134] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] JingsongLi merged pull request #15568: [FLINK-17371][table-runtime] Add cases for decimal casting
JingsongLi merged pull request #15568: URL: https://github.com/apache/flink/pull/15568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-21305) Fix Cumulative and Hopping window should accumulate late events belonging to the cleaned slice
[
https://issues.apache.org/jira/browse/FLINK-21305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-21305.
---
Resolution: Fixed
Fixed in master: 3dd50dd01e49c79d33165a737974cb8e913cb195
> Fix Cumulative and Hopping window should accumulate late events belonging to
> the cleaned slice
> --
>
> Key: FLINK-21305
> URL: https://issues.apache.org/jira/browse/FLINK-21305
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
>Reporter: Jark Wu
>Assignee: Jark Wu
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> Currently, the CUMULATE window drops elements belonging to the cleaned
> slices. This will lead to more inaccurate result than without slicing
> optimization.
> Hopping window has the similar problem, you can reproduce the problem with
> the following test in {{WindowAggregateJsonITCase}}.
> {code:scala}
> @Test
> public void testEventTimeHopWindow2() throws Exception {
> createTestValuesSinkTable("MySink", "name STRING", "cnt BIGINT");
> String jsonPlan =
> tableEnv.getJsonPlan(
> "insert into MySink select\n"
> + " name,\n"
> + " COUNT(*)\n"
> + "FROM TABLE(\n"
> + " HOP(TABLE MyTable, DESCRIPTOR(rowtime),
> INTERVAL '5' SECOND, INTERVAL '10' SECOND))\n"
> + "GROUP BY name, window_start, window_end");
> tableEnv.executeJsonPlan(jsonPlan).await();
> List result = TestValuesTableFactory.getResults("MySink");
> assertResult(
> Arrays.asList(
> "+I[a, 1]",
> "+I[a, 4]",
> "+I[a, 6]",
> "+I[b, 1]",
> "+I[b, 1]",
> "+I[b, 1]",
> "+I[b, 1]",
> "+I[b, 2]",
> "+I[b, 2]",
> "+I[null, 1]",
> "+I[null, 1]"),
> result);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-11838) Create RecoverableWriter for GCS
[ https://issues.apache.org/jira/browse/FLINK-11838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320638#comment-17320638 ] Xintong Song commented on FLINK-11838: -- Thanks [~galenwarren]. I'll try to take a look asap. Just to managed expectation, this could take some time. We are currently prioritizing the 1.13.0 release testing works. > Create RecoverableWriter for GCS > > > Key: FLINK-11838 > URL: https://issues.apache.org/jira/browse/FLINK-11838 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem >Affects Versions: 1.8.0 >Reporter: Fokko Driesprong >Assignee: Galen Warren >Priority: Major > Labels: pull-request-available, usability > Fix For: 1.13.0 > > Time Spent: 20m > Remaining Estimate: 0h > > GCS supports the resumable upload which we can use to create a Recoverable > writer similar to the S3 implementation: > https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload > After using the Hadoop compatible interface: > https://github.com/apache/flink/pull/7519 > We've noticed that the current implementation relies heavily on the renaming > of the files on the commit: > https://github.com/apache/flink/blob/master/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java#L233-L259 > This is suboptimal on an object store such as GCS. Therefore we would like to > implement a more GCS native RecoverableWriter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] wuchong commented on pull request #15495: [FLINK-21305][table-planner-blink] Fix Cumulative and Hopping window should accumulate late events belonging to the cleaned slice
wuchong commented on pull request #15495: URL: https://github.com/apache/flink/pull/15495#issuecomment-819165646 Merging... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wuchong merged pull request #15495: [FLINK-21305][table-planner-blink] Fix Cumulative and Hopping window should accumulate late events belonging to the cleaned slice
wuchong merged pull request #15495: URL: https://github.com/apache/flink/pull/15495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] xintongsong closed pull request #14875: [FLINK-11838][flink-gs-fs-hadoop] Add GCS FileSystem with RecoverableWriter
xintongsong closed pull request #14875: URL: https://github.com/apache/flink/pull/14875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15601: [FLINK-18071][FLINK-21996][coordination] - Part two: Ensure reliable OperatorEvent to running Task matching
flinkbot edited a comment on pull request #15601: URL: https://github.com/apache/flink/pull/15601#issuecomment-819126497 ## CI report: * 86aa312aaae52cfe5529003c9cb50f4b7b824c64 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16485) * 6e5b9772808314793e666e5e2b9f2953dad30edd Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16486) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] KurtYoung commented on pull request #15564: [FLINK-22207][connectors/hive]Hive Catalog retrieve Flink Properties …
KurtYoung commented on pull request #15564: URL: https://github.com/apache/flink/pull/15564#issuecomment-819161036 [ERROR] Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.4.2:check (spotless-check) on project flink-connector-hive_2.11: The following files had format violations: [ERROR] src/test/java/org/apache/flink/table/catalog/hive/HiveCatalogTest.java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20779) Add documentation for row-based operation in Python Table API
[ https://issues.apache.org/jira/browse/FLINK-20779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320631#comment-17320631 ] Huang Xingbo commented on FLINK-20779: -- Hi [~dwysakowicz], I'm working this and will open a relevant PR today. > Add documentation for row-based operation in Python Table API > - > > Key: FLINK-20779 > URL: https://issues.apache.org/jira/browse/FLINK-20779 > Project: Flink > Issue Type: Sub-task > Components: API / Python, Documentation >Affects Versions: 1.13.0 >Reporter: Dian Fu >Assignee: Huang Xingbo >Priority: Major > Fix For: 1.13.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-20779) Add documentation for row-based operation in Python Table API
[ https://issues.apache.org/jira/browse/FLINK-20779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320631#comment-17320631 ] Huang Xingbo edited comment on FLINK-20779 at 4/14/21, 1:41 AM: Hi [~dwysakowicz], I'm working on this and will open a relevant PR today. was (Author: hxbks2ks): Hi [~dwysakowicz], I'm working this and will open a relevant PR today. > Add documentation for row-based operation in Python Table API > - > > Key: FLINK-20779 > URL: https://issues.apache.org/jira/browse/FLINK-20779 > Project: Flink > Issue Type: Sub-task > Components: API / Python, Documentation >Affects Versions: 1.13.0 >Reporter: Dian Fu >Assignee: Huang Xingbo >Priority: Major > Fix For: 1.13.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-22261) Python StreamingModeDataStreamTests is failed on Azure
[
https://issues.apache.org/jira/browse/FLINK-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu closed FLINK-22261.
---
Resolution: Duplicate
This issue should have been fixed in FLINK-22191.
> Python StreamingModeDataStreamTests is failed on Azure
> --
>
> Key: FLINK-22261
> URL: https://issues.apache.org/jira/browse/FLINK-22261
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.13.0
>Reporter: Jark Wu
>Priority: Major
> Fix For: 1.13.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16443=logs=9cada3cb-c1d3-5621-16da-0f718fb86602=8d78fe4f-d658-5c70-12f8-4921589024c3
> {code}
> 2021-04-13T11:49:32.1640428Z === FAILURES
> ===
> 2021-04-13T11:49:32.1641478Z _
> StreamingModeDataStreamTests.test_keyed_process_function_with_state __
> 2021-04-13T11:49:32.1641744Z
> 2021-04-13T11:49:32.1642074Z self =
> testMethod=test_keyed_process_function_with_state>
> 2021-04-13T11:49:32.1642359Z
> 2021-04-13T11:49:32.1642606Z def
> test_keyed_process_function_with_state(self):
> 2021-04-13T11:49:32.1644412Z self.env.set_parallelism(1)
> 2021-04-13T11:49:32.1644941Z
> self.env.get_config().set_auto_watermark_interval(2000)
> 2021-04-13T11:49:32.1645447Z
> self.env.set_stream_time_characteristic(TimeCharacteristic.EventTime)
> 2021-04-13T11:49:32.1647182Z data_stream =
> self.env.from_collection([(1, 'hi', '1603708211000'),
> 2021-04-13T11:49:32.1648276Z
> (2, 'hello', '1603708224000'),
> 2021-04-13T11:49:32.1661775Z
> (3, 'hi', '1603708226000'),
> 2021-04-13T11:49:32.1663379Z
> (4, 'hello', '1603708289000'),
> 2021-04-13T11:49:32.1665197Z
> (5, 'hi', '1603708291000'),
> 2021-04-13T11:49:32.1666200Z
> (6, 'hello', '1603708293000')],
> 2021-04-13T11:49:32.1666827Z
> type_info=Types.ROW([Types.INT(), Types.STRING(),
> 2021-04-13T11:49:32.1667449Z
>Types.STRING()]))
> 2021-04-13T11:49:32.1667830Z
> 2021-04-13T11:49:32.1668351Z class
> MyTimestampAssigner(TimestampAssigner):
> 2021-04-13T11:49:32.1668755Z
> 2021-04-13T11:49:32.1669783Z def extract_timestamp(self, value,
> record_timestamp) -> int:
> 2021-04-13T11:49:32.1670386Z return int(value[2])
> 2021-04-13T11:49:32.1670672Z
> 2021-04-13T11:49:32.1671063Z class
> MyProcessFunction(KeyedProcessFunction):
> 2021-04-13T11:49:32.1671434Z
> 2021-04-13T11:49:32.1671727Z def __init__(self):
> 2021-04-13T11:49:32.1672090Z self.value_state = None
> 2021-04-13T11:49:32.1685812Z self.list_state = None
> 2021-04-13T11:49:32.1686276Z self.map_state = None
> 2021-04-13T11:49:32.1686609Z
> 2021-04-13T11:49:32.1687039Z def open(self, runtime_context:
> RuntimeContext):
> 2021-04-13T11:49:32.1688350Z value_state_descriptor =
> ValueStateDescriptor('value_state', Types.INT())
> 2021-04-13T11:49:32.1688953Z self.value_state =
> runtime_context.get_state(value_state_descriptor)
> 2021-04-13T11:49:32.1689892Z list_state_descriptor =
> ListStateDescriptor('list_state', Types.INT())
> 2021-04-13T11:49:32.1690492Z self.list_state =
> runtime_context.get_list_state(list_state_descriptor)
> 2021-04-13T11:49:32.1691407Z map_state_descriptor =
> MapStateDescriptor('map_state', Types.INT(), Types.STRING())
> 2021-04-13T11:49:32.1692052Z self.map_state =
> runtime_context.get_map_state(map_state_descriptor)
> 2021-04-13T11:49:32.1692481Z
> 2021-04-13T11:49:32.1693134Z def process_element(self, value,
> ctx):
> 2021-04-13T11:49:32.1693632Z current_value =
> self.value_state.value()
> 2021-04-13T11:49:32.1694106Z self.value_state.update(value[0])
> 2021-04-13T11:49:32.1694573Z current_list = [_ for _ in
> self.list_state.get()]
> 2021-04-13T11:49:32.1695051Z self.list_state.add(value[0])
> 2021-04-13T11:49:32.1695445Z map_entries_string = []
> 2021-04-13T11:49:32.1695902Z for k, v in
> self.map_state.items():
> 2021-04-13T11:49:32.1696822Z
> map_entries_string.append(str(k) + ': ' + str(v))
> 2021-04-13T11:49:32.1697700Z map_entries_string = '{' + ',
>
[jira] [Updated] (FLINK-22261) Python StreamingModeDataStreamTests is failed on Azure
[
https://issues.apache.org/jira/browse/FLINK-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu updated FLINK-22261:
Fix Version/s: (was: 1.13.0)
> Python StreamingModeDataStreamTests is failed on Azure
> --
>
> Key: FLINK-22261
> URL: https://issues.apache.org/jira/browse/FLINK-22261
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.13.0
>Reporter: Jark Wu
>Priority: Major
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16443=logs=9cada3cb-c1d3-5621-16da-0f718fb86602=8d78fe4f-d658-5c70-12f8-4921589024c3
> {code}
> 2021-04-13T11:49:32.1640428Z === FAILURES
> ===
> 2021-04-13T11:49:32.1641478Z _
> StreamingModeDataStreamTests.test_keyed_process_function_with_state __
> 2021-04-13T11:49:32.1641744Z
> 2021-04-13T11:49:32.1642074Z self =
> testMethod=test_keyed_process_function_with_state>
> 2021-04-13T11:49:32.1642359Z
> 2021-04-13T11:49:32.1642606Z def
> test_keyed_process_function_with_state(self):
> 2021-04-13T11:49:32.1644412Z self.env.set_parallelism(1)
> 2021-04-13T11:49:32.1644941Z
> self.env.get_config().set_auto_watermark_interval(2000)
> 2021-04-13T11:49:32.1645447Z
> self.env.set_stream_time_characteristic(TimeCharacteristic.EventTime)
> 2021-04-13T11:49:32.1647182Z data_stream =
> self.env.from_collection([(1, 'hi', '1603708211000'),
> 2021-04-13T11:49:32.1648276Z
> (2, 'hello', '1603708224000'),
> 2021-04-13T11:49:32.1661775Z
> (3, 'hi', '1603708226000'),
> 2021-04-13T11:49:32.1663379Z
> (4, 'hello', '1603708289000'),
> 2021-04-13T11:49:32.1665197Z
> (5, 'hi', '1603708291000'),
> 2021-04-13T11:49:32.1666200Z
> (6, 'hello', '1603708293000')],
> 2021-04-13T11:49:32.1666827Z
> type_info=Types.ROW([Types.INT(), Types.STRING(),
> 2021-04-13T11:49:32.1667449Z
>Types.STRING()]))
> 2021-04-13T11:49:32.1667830Z
> 2021-04-13T11:49:32.1668351Z class
> MyTimestampAssigner(TimestampAssigner):
> 2021-04-13T11:49:32.1668755Z
> 2021-04-13T11:49:32.1669783Z def extract_timestamp(self, value,
> record_timestamp) -> int:
> 2021-04-13T11:49:32.1670386Z return int(value[2])
> 2021-04-13T11:49:32.1670672Z
> 2021-04-13T11:49:32.1671063Z class
> MyProcessFunction(KeyedProcessFunction):
> 2021-04-13T11:49:32.1671434Z
> 2021-04-13T11:49:32.1671727Z def __init__(self):
> 2021-04-13T11:49:32.1672090Z self.value_state = None
> 2021-04-13T11:49:32.1685812Z self.list_state = None
> 2021-04-13T11:49:32.1686276Z self.map_state = None
> 2021-04-13T11:49:32.1686609Z
> 2021-04-13T11:49:32.1687039Z def open(self, runtime_context:
> RuntimeContext):
> 2021-04-13T11:49:32.1688350Z value_state_descriptor =
> ValueStateDescriptor('value_state', Types.INT())
> 2021-04-13T11:49:32.1688953Z self.value_state =
> runtime_context.get_state(value_state_descriptor)
> 2021-04-13T11:49:32.1689892Z list_state_descriptor =
> ListStateDescriptor('list_state', Types.INT())
> 2021-04-13T11:49:32.1690492Z self.list_state =
> runtime_context.get_list_state(list_state_descriptor)
> 2021-04-13T11:49:32.1691407Z map_state_descriptor =
> MapStateDescriptor('map_state', Types.INT(), Types.STRING())
> 2021-04-13T11:49:32.1692052Z self.map_state =
> runtime_context.get_map_state(map_state_descriptor)
> 2021-04-13T11:49:32.1692481Z
> 2021-04-13T11:49:32.1693134Z def process_element(self, value,
> ctx):
> 2021-04-13T11:49:32.1693632Z current_value =
> self.value_state.value()
> 2021-04-13T11:49:32.1694106Z self.value_state.update(value[0])
> 2021-04-13T11:49:32.1694573Z current_list = [_ for _ in
> self.list_state.get()]
> 2021-04-13T11:49:32.1695051Z self.list_state.add(value[0])
> 2021-04-13T11:49:32.1695445Z map_entries_string = []
> 2021-04-13T11:49:32.1695902Z for k, v in
> self.map_state.items():
> 2021-04-13T11:49:32.1696822Z
> map_entries_string.append(str(k) + ': ' + str(v))
> 2021-04-13T11:49:32.1697700Z map_entries_string = '{' + ',
> '.join(map_entries_string) + '}'
> 2021-04-13T11:49:32.1698483Z
