[GitHub] [flink] flinkbot edited a comment on pull request #12422: [FLINK-17901][python] Add module interface in PyFlink
flinkbot edited a comment on pull request #12422: URL: https://github.com/apache/flink/pull/12422#issuecomment-636647115 ## CI report: * 3a387da20f29d9b1f6baf1e57c6d9979d27dd81f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2509) * a57beb047e52bacd60d9c19cbcd31bc9b1d88ec4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2599) * 2a1da9cc0e95b2a429a577fd83955380e30e323c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-18082) UnsignedTypeConversionITCase stalls in ch.vorburger.mariadb4j.DB.stop
Robert Metzger created FLINK-18082:
--
Summary: UnsignedTypeConversionITCase stalls in
ch.vorburger.mariadb4j.DB.stop
Key: FLINK-18082
URL: https://issues.apache.org/jira/browse/FLINK-18082
Project: Flink
Issue Type: Bug
Components: Connectors / JDBC
Affects Versions: 1.12.0
Reporter: Robert Metzger
CI:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2582=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20
{code}
2020-06-02T19:01:31.8486456Z
==
2020-06-02T19:01:31.8487052Z Printing stack trace of Java process 8653
2020-06-02T19:01:31.8487424Z
==
2020-06-02T19:01:31.8541169Z Picked up JAVA_TOOL_OPTIONS:
-XX:+HeapDumpOnOutOfMemoryError
2020-06-02T19:01:32.1665740Z 2020-06-02 19:01:32
2020-06-02T19:01:32.1666470Z Full thread dump OpenJDK 64-Bit Server VM
(25.242-b08 mixed mode):
2020-06-02T19:01:32.1666735Z
2020-06-02T19:01:32.1667614Z "Attach Listener" #537 daemon prio=9 os_prio=0
tid=0x7f61f8001000 nid=0x3b9f waiting on condition [0x]
2020-06-02T19:01:32.1668130Zjava.lang.Thread.State: RUNNABLE
2020-06-02T19:01:32.1668311Z
2020-06-02T19:01:32.1668958Z "flink-akka.actor.default-dispatcher-193" #535
prio=5 os_prio=0 tid=0x7f6034001000 nid=0x3af7 waiting on condition
[0x7f61a25b8000]
2020-06-02T19:01:32.1669418Zjava.lang.Thread.State: TIMED_WAITING (parking)
2020-06-02T19:01:32.1669730Zat sun.misc.Unsafe.park(Native Method)
2020-06-02T19:01:32.1670301Z- parking to wait for <0x80c51528> (a
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
2020-06-02T19:01:32.1670791Zat
akka.dispatch.forkjoin.ForkJoinPool.idleAwaitWork(ForkJoinPool.java:2135)
2020-06-02T19:01:32.1671329Zat
akka.dispatch.forkjoin.ForkJoinPool.scan(ForkJoinPool.java:2067)
2020-06-02T19:01:32.1671763Zat
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
2020-06-02T19:01:32.1672211Zat
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2020-06-02T19:01:32.1672491Z
2020-06-02T19:01:32.1673104Z "flink-akka.actor.default-dispatcher-191" #533
prio=5 os_prio=0 tid=0x7f619801e000 nid=0x3ae1 waiting on condition
[0x7f60770f1000]
2020-06-02T19:01:32.1673564Zjava.lang.Thread.State: WAITING (parking)
2020-06-02T19:01:32.1673839Zat sun.misc.Unsafe.park(Native Method)
2020-06-02T19:01:32.1674422Z- parking to wait for <0x80c51528> (a
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
2020-06-02T19:01:32.1674865Zat
akka.dispatch.forkjoin.ForkJoinPool.scan(ForkJoinPool.java:2075)
2020-06-02T19:01:32.1675305Zat
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
2020-06-02T19:01:32.1675751Zat
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2020-06-02T19:01:32.1676046Z
2020-06-02T19:01:32.1676669Z "jobmanager-future-thread-2" #466 daemon prio=5
os_prio=0 tid=0x7f6124001000 nid=0x3795 waiting on condition
[0x7f61a23b6000]
2020-06-02T19:01:32.1677316Zjava.lang.Thread.State: WAITING (parking)
2020-06-02T19:01:32.1677617Zat sun.misc.Unsafe.park(Native Method)
2020-06-02T19:01:32.1678220Z- parking to wait for <0x816a4c90> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
2020-06-02T19:01:32.1678702Zat
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
2020-06-02T19:01:32.1679209Zat
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
2020-06-02T19:01:32.1679822Zat
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
2020-06-02T19:01:32.1680422Zat
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
2020-06-02T19:01:32.1680962Zat
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
2020-06-02T19:01:32.1681424Zat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
2020-06-02T19:01:32.1682062Zat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2020-06-02T19:01:32.1682445Zat java.lang.Thread.run(Thread.java:748)
2020-06-02T19:01:32.1682656Z
2020-06-02T19:01:32.1683271Z "Flink-DispatcherRestEndpoint-thread-4" #349
daemon prio=5 os_prio=0 tid=0x7f618c00a000 nid=0x29a4 waiting on condition
[0x7f61a029f000]
2020-06-02T19:01:32.1683750Zjava.lang.Thread.State: TIMED_WAITING (parking)
2020-06-02T19:01:32.1684057Zat sun.misc.Unsafe.park(Native Method)
2020-06-02T19:01:32.1684648Z- parking to wait for <0x81731ff8>
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434322529
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
} else if (operation instanceof UseDatabaseOperation) {
cmd = SqlCommand.USE;
- UseDatabaseOperation op = ((UseDatabaseOperation)
operation);
- operands = new String[] { String.format("`%s`.`%s`",
op.getCatalogName(), op.getDatabaseName()) };
+ operands = new String[] { ((UseDatabaseOperation)
operation).getDatabaseName() };
Review comment:
catalog name is unnecessary, only database is needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434321402
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ("\`") is skipped by sql parser, so we need not to
handle this. see the test (TestItem.validSql("use \`default\`;",
SqlCommand.USE, "default")) in `SqlCommandParserTest#testCommands`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434321402
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string \` is skipped by sql parser, so we need not to
handle this. see the test (`TestItem.validSql("use `default`;", SqlCommand.USE,
"default")`) in `SqlCommandParserTest#testCommands`
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ("\`") is skipped by sql parser, so we need not to
handle this. see the test (`TestItem.validSql("use `default`;", SqlCommand.USE,
"default")`) in `SqlCommandParserTest#testCommands`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434321402
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ("\`") is skipped by sql parser, so we need not to
handle this. see the test (`TestItem.validSql("use \`default\`;",
SqlCommand.USE, "default")`) in `SqlCommandParserTest#testCommands`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-18081) Fix broken links in "Kerberos Authentication Setup and Configuration" doc
[
https://issues.apache.org/jira/browse/FLINK-18081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124627#comment-17124627
]
Yangze Guo commented on FLINK-18081:
Could someone assign this to me?
> Fix broken links in "Kerberos Authentication Setup and Configuration" doc
> -
>
> Key: FLINK-18081
> URL: https://issues.apache.org/jira/browse/FLINK-18081
> Project: Flink
> Issue Type: Bug
>Reporter: Yangze Guo
>Priority: Major
>
> The {{config.html#kerberos-based-security}} is not valid now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-18081) Fix broken links in "Kerberos Authentication Setup and Configuration" doc
Yangze Guo created FLINK-18081:
--
Summary: Fix broken links in "Kerberos Authentication Setup and
Configuration" doc
Key: FLINK-18081
URL: https://issues.apache.org/jira/browse/FLINK-18081
Project: Flink
Issue Type: Bug
Reporter: Yangze Guo
The {{config.html#kerberos-based-security}} is not valid now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434321402
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ` is skipped by sql parser, so we need not to handle
this. see the test (`TestItem.validSql("use `default`;", SqlCommand.USE,
"default")`) in `SqlCommandParserTest#testCommands`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434321402
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ` \` ` is skipped by sql parser, so we need not to
handle this. see the test (`TestItem.validSql("use `default`;", SqlCommand.USE,
"default")`) in `SqlCommandParserTest#testCommands`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434321402
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ` "`" ` is skipped by sql parser, so we need not to
handle this. see the test (`TestItem.validSql("use `default`;", SqlCommand.USE,
"default")`) in `SqlCommandParserTest#testCommands`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434321402
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ``` is skipped by sql parser, so we need not to
handle this. see the test (`TestItem.validSql("use `default`;", SqlCommand.USE,
"default")`) in `SqlCommandParserTest#testCommands`
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
The quoting string ` ` ` is skipped by sql parser, so we need not to
handle this. see the test (`TestItem.validSql("use `default`;", SqlCommand.USE,
"default")`) in `SqlCommandParserTest#testCommands`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-17851) FlinkKafkaProducerITCase.testResumeTransaction:108->assertRecord NoSuchElement
[
https://issues.apache.org/jira/browse/FLINK-17851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124623#comment-17124623
]
Robert Metzger commented on FLINK-17851:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2586=logs=4be4ed2b-549a-533d-aa33-09e28e360cc8=0db94045-2aa0-53fa-f444-0130d6933518
> FlinkKafkaProducerITCase.testResumeTransaction:108->assertRecord NoSuchElement
> --
>
> Key: FLINK-17851
> URL: https://issues.apache.org/jira/browse/FLINK-17851
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Tests
>Affects Versions: 1.11.0
>Reporter: Robert Metzger
>Priority: Critical
> Labels: test-stability
> Fix For: 1.11.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=1957=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=34f486e1-e1e4-5dd2-9c06-bfdd9b9c74a8
> {code}
> 2020-05-20T16:36:17.1033719Z [ERROR] Tests run: 10, Failures: 0, Errors: 1,
> Skipped: 0, Time elapsed: 68.936 s <<< FAILURE! - in
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase
> 2020-05-20T16:36:17.1035011Z [ERROR]
> testResumeTransaction(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase)
> Time elapsed: 11.918 s <<< ERROR!
> 2020-05-20T16:36:17.1036296Z java.util.NoSuchElementException
> 2020-05-20T16:36:17.1036802Z at
> org.apache.kafka.common.utils.AbstractIterator.next(AbstractIterator.java:52)
> 2020-05-20T16:36:17.1037553Z at
> org.apache.flink.shaded.guava18.com.google.common.collect.Iterators.getOnlyElement(Iterators.java:302)
> 2020-05-20T16:36:17.1038087Z at
> org.apache.flink.shaded.guava18.com.google.common.collect.Iterables.getOnlyElement(Iterables.java:289)
> 2020-05-20T16:36:17.1038654Z at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.assertRecord(FlinkKafkaProducerITCase.java:201)
> 2020-05-20T16:36:17.1039239Z at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.testResumeTransaction(FlinkKafkaProducerITCase.java:108)
> 2020-05-20T16:36:17.1039727Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-05-20T16:36:17.1040131Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-05-20T16:36:17.1040575Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-05-20T16:36:17.1040989Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-05-20T16:36:17.1041515Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2020-05-20T16:36:17.1041989Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2020-05-20T16:36:17.1042468Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2020-05-20T16:36:17.1042908Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2020-05-20T16:36:17.1043497Z at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> 2020-05-20T16:36:17.1044096Z at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> 2020-05-20T16:36:17.1044833Z at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 2020-05-20T16:36:17.1045171Z at java.lang.Thread.run(Thread.java:748)
> 2020-05-20T16:36:17.1045358Z
> 2020-05-20T16:36:17.6145018Z [INFO]
> 2020-05-20T16:36:17.6145477Z [INFO] Results:
> 2020-05-20T16:36:17.6145654Z [INFO]
> 2020-05-20T16:36:17.6145838Z [ERROR] Errors:
> 2020-05-20T16:36:17.6146898Z [ERROR]
> FlinkKafkaProducerITCase.testResumeTransaction:108->assertRecord:201 »
> NoSuchElement
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-18037) The doc of StreamTaskNetworkInput.java may has a redundant 'status'
[
https://issues.apache.org/jira/browse/FLINK-18037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124620#comment-17124620
]
Jark Wu commented on FLINK-18037:
-
I think so.
> The doc of StreamTaskNetworkInput.java may has a redundant 'status'
> ---
>
> Key: FLINK-18037
> URL: https://issues.apache.org/jira/browse/FLINK-18037
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.10.0, 1.10.1, 1.11.0
>Reporter: ZhuShang
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The doc of class StreamTaskNetworkInput as follows
> {code:java}
> * Forwarding elements, watermarks, or status status elements must be
> protected by synchronizing
> * on the given lock object. This ensures that we don't call methods on a
> * {@link StreamInputProcessor} concurrently with the timer callback or other
> things.{code}
>
> Is one of the 'status' redundant?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-17260) StreamingKafkaITCase failure on Azure
[
https://issues.apache.org/jira/browse/FLINK-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124621#comment-17124621
]
Robert Metzger commented on FLINK-17260:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2586=logs=08866332-78f7-59e4-4f7e-49a56faa3179=931b3127-d6ee-5f94-e204-48d51cd1c334
> StreamingKafkaITCase failure on Azure
> -
>
> Key: FLINK-17260
> URL: https://issues.apache.org/jira/browse/FLINK-17260
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Tests
>Affects Versions: 1.11.0, 1.12.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.11.0
>
>
> [https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_apis/build/builds/7544/logs/165]
>
> {code:java}
> 2020-04-16T00:12:32.2848429Z [INFO] Running
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-04-16T00:14:47.9100927Z [ERROR] Tests run: 3, Failures: 1, Errors: 0,
> Skipped: 0, Time elapsed: 135.621 s <<< FAILURE! - in
> org.apache.flink.tests.util.k afka.StreamingKafkaITCase
> 2020-04-16T00:14:47.9103036Z [ERROR] testKafka[0:
> kafka-version:0.10.2.0](org.apache.flink.tests.util.kafka.StreamingKafkaITCase)
> Time elapsed: 46.222 s <<< FAILURE!
> 2020-04-16T00:14:47.9104033Z java.lang.AssertionError:
> expected:<[elephant,27,64213]> but was:<[]>
> 2020-04-16T00:14:47.9104638Zat org.junit.Assert.fail(Assert.java:88)
> 2020-04-16T00:14:47.9105148Zat
> org.junit.Assert.failNotEquals(Assert.java:834)
> 2020-04-16T00:14:47.9105701Zat
> org.junit.Assert.assertEquals(Assert.java:118)
> 2020-04-16T00:14:47.9106239Zat
> org.junit.Assert.assertEquals(Assert.java:144)
> 2020-04-16T00:14:47.9107177Zat
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase.testKafka(StreamingKafkaITCase.java:162)
> 2020-04-16T00:14:47.9107845Zat
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-04-16T00:14:47.9108434Zat
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-04-16T00:14:47.9109318Zat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-04-16T00:14:47.9109914Zat
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-04-16T00:14:47.9110434Zat
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2020-04-16T00:14:47.9110985Zat
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2020-04-16T00:14:47.9111548Zat
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2020-04-16T00:14:47.9112083Zat
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2020-04-16T00:14:47.9112629Zat
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:48)
> 2020-04-16T00:14:47.9113145Zat
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:48)
> 2020-04-16T00:14:47.9113637Zat
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2020-04-16T00:14:47.9114072Zat
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2020-04-16T00:14:47.9114490Zat
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2020-04-16T00:14:47.9115256Zat
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2020-04-16T00:14:47.9115791Zat
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2020-04-16T00:14:47.9116292Zat
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-04-16T00:14:47.9116736Zat
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-04-16T00:14:47.9117779Zat
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-04-16T00:14:47.9118274Zat
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-04-16T00:14:47.9118766Zat
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2020-04-16T00:14:47.9119204Zat
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-04-16T00:14:47.9119625Zat
> org.junit.runners.Suite.runChild(Suite.java:128)
> 2020-04-16T00:14:47.9120005Zat
> org.junit.runners.Suite.runChild(Suite.java:27)
> 2020-04-16T00:14:47.9120428Zat
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-04-16T00:14:47.9120876Zat
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-04-16T00:14:47.9121350Zat
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-04-16T00:14:47.9121805Zat
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-04-16T00:14:47.9122273Zat
>
[jira] [Assigned] (FLINK-18037) The doc of StreamTaskNetworkInput.java may has a redundant 'status'
[
https://issues.apache.org/jira/browse/FLINK-18037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu reassigned FLINK-18037:
---
Assignee: ZhuShang
> The doc of StreamTaskNetworkInput.java may has a redundant 'status'
> ---
>
> Key: FLINK-18037
> URL: https://issues.apache.org/jira/browse/FLINK-18037
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.10.0, 1.10.1, 1.11.0
>Reporter: ZhuShang
>Assignee: ZhuShang
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The doc of class StreamTaskNetworkInput as follows
> {code:java}
> * Forwarding elements, watermarks, or status status elements must be
> protected by synchronizing
> * on the given lock object. This ensures that we don't call methods on a
> * {@link StreamInputProcessor} concurrently with the timer callback or other
> things.{code}
>
> Is one of the 'status' redundant?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-17824) "Resuming Savepoint" e2e stalls indefinitely
[
https://issues.apache.org/jira/browse/FLINK-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124618#comment-17124618
]
Robert Metzger commented on FLINK-17824:
Now with logs:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2586=logs=91bf6583-3fb2-592f-e4d4-d79d79c3230a=94459a52-42b6-5bfc-5d74-690b5d3c6de8
(hadoop 3.1.3 profile, see FLINK-17404)
> "Resuming Savepoint" e2e stalls indefinitely
> -
>
> Key: FLINK-17824
> URL: https://issues.apache.org/jira/browse/FLINK-17824
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Tests
>Affects Versions: 1.12.0
>Reporter: Robert Metzger
>Assignee: Robert Metzger
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.12.0
>
>
> CI;
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=1887=logs=91bf6583-3fb2-592f-e4d4-d79d79c3230a=94459a52-42b6-5bfc-5d74-690b5d3c6de8
> {code}
> 2020-05-19T21:05:52.9696236Z
> ==
> 2020-05-19T21:05:52.9696860Z Running 'Resuming Savepoint (file, async, scale
> down) end-to-end test'
> 2020-05-19T21:05:52.9697243Z
> ==
> 2020-05-19T21:05:52.9713094Z TEST_DATA_DIR:
> /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/temp-test-directory-52970362751
> 2020-05-19T21:05:53.1194478Z Flink dist directory:
> /home/vsts/work/1/s/flink-dist/target/flink-1.12-SNAPSHOT-bin/flink-1.12-SNAPSHOT
> 2020-05-19T21:05:53.2180375Z Starting cluster.
> 2020-05-19T21:05:53.9986167Z Starting standalonesession daemon on host
> fv-az558.
> 2020-05-19T21:05:55.5997224Z Starting taskexecutor daemon on host fv-az558.
> 2020-05-19T21:05:55.6223837Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-19T21:05:57.0552482Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-19T21:05:57.9446865Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-19T21:05:59.0098434Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-19T21:06:00.0569710Z Dispatcher REST endpoint is up.
> 2020-05-19T21:06:07.7099937Z Job (a92a74de8446a80403798bb4806b73f3) is
> running.
> 2020-05-19T21:06:07.7855906Z Waiting for job to process up to 200 records,
> current progress: 114 records ...
> 2020-05-19T21:06:55.5755111Z
> 2020-05-19T21:06:55.5756550Z
>
> 2020-05-19T21:06:55.5757225Z The program finished with the following
> exception:
> 2020-05-19T21:06:55.5757566Z
> 2020-05-19T21:06:55.5765453Z org.apache.flink.util.FlinkException: Could not
> stop with a savepoint job "a92a74de8446a80403798bb4806b73f3".
> 2020-05-19T21:06:55.5766873Z at
> org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:485)
> 2020-05-19T21:06:55.5767980Z at
> org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:854)
> 2020-05-19T21:06:55.5769014Z at
> org.apache.flink.client.cli.CliFrontend.stop(CliFrontend.java:477)
> 2020-05-19T21:06:55.5770052Z at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:921)
> 2020-05-19T21:06:55.5771107Z at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:982)
> 2020-05-19T21:06:55.5772223Z at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> 2020-05-19T21:06:55.5773325Z at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:982)
> 2020-05-19T21:06:55.5774871Z Caused by:
> java.util.concurrent.ExecutionException:
> java.util.concurrent.CompletionException:
> java.util.concurrent.CompletionException:
> org.apache.flink.runtime.checkpoint.CheckpointException: Checkpoint
> Coordinator is suspending.
> 2020-05-19T21:06:55.5777183Z at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> 2020-05-19T21:06:55.5778884Z at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
> 2020-05-19T21:06:55.5779920Z at
> org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:483)
> 2020-05-19T21:06:55.5781175Z ... 6 more
> 2020-05-19T21:06:55.5782391Z Caused by:
> java.util.concurrent.CompletionException:
> java.util.concurrent.CompletionException:
> org.apache.flink.runtime.checkpoint.CheckpointException: Checkpoint
> Coordinator is suspending.
> 2020-05-19T21:06:55.5783885Z at
> org.apache.flink.runtime.scheduler.SchedulerBase.lambda$stopWithSavepoint$9(SchedulerBase.java:890)
> 2020-05-19T21:06:55.5784992Z at
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)
> 2020-05-19T21:06:55.5786492Z at
>
[GitHub] [flink] wuchong commented on a change in pull request #12436: [FLINK-17847][table sql / planner] ArrayIndexOutOfBoundsException happens in StreamExecCalc operator
wuchong commented on a change in pull request #12436:
URL: https://github.com/apache/flink/pull/12436#discussion_r434317766
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala
##
@@ -1643,19 +1644,32 @@ object ScalarOperatorGens {
val resultTypeTerm = primitiveTypeTermForType(componentInfo)
val defaultTerm = primitiveDefaultValue(componentInfo)
+if (index.literalValue.isDefined &&
+index.literalValue.get.isInstanceOf[Int] &&
+index.literalValue.get.asInstanceOf[Int] < 1) {
+ throw new ValidationException(s"Array element access needs an index
starting at 1 but was " +
Review comment:
Can be simplified to
```scala
index.literalValue match {
case Some(v: Int) if v < 1 =>
throw new ValidationException(
s"Array element access needs an index starting at 1 but was $v")
case _ => // nothing
}
```
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala
##
@@ -1643,19 +1644,32 @@ object ScalarOperatorGens {
val resultTypeTerm = primitiveTypeTermForType(componentInfo)
val defaultTerm = primitiveDefaultValue(componentInfo)
+if (index.literalValue.isDefined &&
+index.literalValue.get.isInstanceOf[Int] &&
+index.literalValue.get.asInstanceOf[Int] < 1) {
+ throw new ValidationException(s"Array element access needs an index
starting at 1 but was " +
+s"${index.literalValue.get.asInstanceOf[Int]}.")
+}
val idxStr = s"${index.resultTerm} - 1"
val arrayIsNull = s"${array.resultTerm}.isNullAt($idxStr)"
val arrayGet =
rowFieldReadAccess(ctx, idxStr, array.resultTerm, componentInfo)
+/**
+ * Return null when array index out of bounds which follows Calcite's
behaviour.
+ * @see org.apache.calcite.sql.fun.SqlStdOperatorTable
Review comment:
`* @see [[org.apache.calcite.sql.fun.SqlStdOperatorTable.ITEM]]`
Could you move this Javadoc to the above of method signature? Javadoc is
usually not used in code.
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala
##
@@ -1643,19 +1644,35 @@ object ScalarOperatorGens {
val resultTypeTerm = primitiveTypeTermForType(componentInfo)
val defaultTerm = primitiveDefaultValue(componentInfo)
+if (index.literalValue.isDefined &&
+index.literalValue.get.isInstanceOf[Int] &&
+index.literalValue.get.asInstanceOf[Int] < 1) {
+ throw new ValidationException(s"Array element access needs an index
starting at 1 but was " +
+s"${index.literalValue.get.asInstanceOf[Int]}.")
+}
val idxStr = s"${index.resultTerm} - 1"
val arrayIsNull = s"${array.resultTerm}.isNullAt($idxStr)"
val arrayGet =
rowFieldReadAccess(ctx, idxStr, array.resultTerm, componentInfo)
+/**
+ * Return null when array index out of bounds which follows Calcite's
behaviour.
+ * @see org.apache.calcite.sql.fun.SqlStdOperatorTable
+ */
val arrayAccessCode =
- s"""
- |${array.code}
- |${index.code}
- |boolean $nullTerm = ${array.nullTerm} || ${index.nullTerm} ||
$arrayIsNull;
- |$resultTypeTerm $resultTerm = $nullTerm ? $defaultTerm : $arrayGet;
- |""".stripMargin
-
+s"""
+|${array.code}
+|${index.code}
+|$resultTypeTerm $resultTerm;
+|boolean $nullTerm;
+|if (${idxStr} < 0 || ${idxStr} >= ${array.resultTerm}.size()) {
Review comment:
I got what's your concern @libenchao . I think we don't have divergency
on the runtime null behavior, the only concern is the compile exception. But I
still prefer the current way, because it's very likely a typo if `my_arr[0]` is
used, so a hint exception is more useful than surprising result.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-15318) RocksDBWriteBatchPerformanceTest.benchMark fails on ppc64le
[ https://issues.apache.org/jira/browse/FLINK-15318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124616#comment-17124616 ] Robert Metzger commented on FLINK-15318: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2587=logs=f0ac5c25-1168-55a5-07ff-0e88223afed9=39a61cac-5c62-532f-d2c1-dea450a66708 > RocksDBWriteBatchPerformanceTest.benchMark fails on ppc64le > --- > > Key: FLINK-15318 > URL: https://issues.apache.org/jira/browse/FLINK-15318 > Project: Flink > Issue Type: Bug > Components: Benchmarks, Runtime / State Backends > Environment: arch: ppc64le > os: rhel7.6, ubuntu 18.04 > jdk: 8, 11 > mvn: 3.3.9, 3.6.2 >Reporter: Siddhesh Ghadi >Priority: Major > Attachments: surefire-report.txt > > > RocksDBWriteBatchPerformanceTest.benchMark fails due to TestTimedOut, however > when test-timeout is increased from 2s to 5s in > org/apache/flink/contrib/streaming/state/benchmark/RocksDBWriteBatchPerformanceTest.java:75, > it passes. Is this acceptable solution? > Note: Tests are ran inside a container. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-18071) CoordinatorEventsExactlyOnceITCase.checkListContainsSequence fails on CI
[
https://issues.apache.org/jira/browse/FLINK-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124615#comment-17124615
]
Robert Metzger commented on FLINK-18071:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2587=logs=6bfdaf55-0c08-5e3f-a2d2-2a0285fd41cf=93018cfc-6498-565b-e034-4b68eb90fc80
> CoordinatorEventsExactlyOnceITCase.checkListContainsSequence fails on CI
>
>
> Key: FLINK-18071
> URL: https://issues.apache.org/jira/browse/FLINK-18071
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Tests
>Affects Versions: 1.12.0
>Reporter: Robert Metzger
>Priority: Critical
> Labels: test-stability
> Fix For: 1.12.0
>
>
> CI:
> https://dev.azure.com/georgeryan1322/Flink/_build/results?buildId=330=logs=6e58d712-c5cc-52fb-0895-6ff7bd56c46b=f30a8e80-b2cf-535c-9952-7f521a4ae374
> {code}
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.795
> s <<< FAILURE! - in
> org.apache.flink.runtime.operators.coordination.CoordinatorEventsExactlyOnceITCase
> [ERROR]
> test(org.apache.flink.runtime.operators.coordination.CoordinatorEventsExactlyOnceITCase)
> Time elapsed: 4.647 s <<< FAILURE!
> java.lang.AssertionError: List did not contain expected sequence of 200
> elements, but was: [152, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
> 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
> 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
> 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
> 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
> 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107,
> 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122,
> 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137,
> 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152,
> 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,
> 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182,
> 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197,
> 198, 199]
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.flink.runtime.operators.coordination.CoordinatorEventsExactlyOnceITCase.failList(CoordinatorEventsExactlyOnceITCase.java:160)
> at
> org.apache.flink.runtime.operators.coordination.CoordinatorEventsExactlyOnceITCase.checkListContainsSequence(CoordinatorEventsExactlyOnceITCase.java:148)
> at
> org.apache.flink.runtime.operators.coordination.CoordinatorEventsExactlyOnceITCase.test(CoordinatorEventsExactlyOnceITCase.java:143)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #12144: [FLINK-17384][flink-dist] support read hbase conf dir from flink.conf and change HBaseConfiguration construction.
flinkbot edited a comment on pull request #12144: URL: https://github.com/apache/flink/pull/12144#issuecomment-628475376 ## CI report: * d36b959fb16a91c15babda10dd884cbbdec58420 UNKNOWN * 84f420d9043272150872fe4a32c2eaad2400a750 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2592) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-16694) Resuming Externalized Checkpoint end-to-end test failed on travis
[
https://issues.apache.org/jira/browse/FLINK-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124613#comment-17124613
]
Robert Metzger commented on FLINK-16694:
It happened again, on the Hadoop 3.1.3 profile:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2587=logs=91bf6583-3fb2-592f-e4d4-d79d79c3230a=94459a52-42b6-5bfc-5d74-690b5d3c6de8
Most likely it's the same issue as FLINK-17404.
> Resuming Externalized Checkpoint end-to-end test failed on travis
> -
>
> Key: FLINK-16694
> URL: https://issues.apache.org/jira/browse/FLINK-16694
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics, Tests
>Affects Versions: 1.9.2
>Reporter: Piotr Nowojski
>Assignee: Chesnay Schepler
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.11.0, 1.10.2, 1.9.4
>
>
> Running 'Resuming Externalized Checkpoint (rocks, incremental, scale down)
> end-to-end test' failed on travis (release-1.9 branch) with the error:
> {code:java}
> The job exceeded the maximum log length, and has been terminated.
> {code}
> https://api.travis-ci.org/v3/job/664469537/log.txt
> https://travis-ci.org/github/apache/flink/builds/664469494
> This error is probably because of metrics logging and it's masking some other
> underlying issue, potentially FLINK-16695 as they happened in the same build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-18075) Kafka connector does not call open method of (de)serialization schema
[ https://issues.apache.org/jira/browse/FLINK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-18075: --- Fix Version/s: 1.11.0 > Kafka connector does not call open method of (de)serialization schema > - > > Key: FLINK-18075 > URL: https://issues.apache.org/jira/browse/FLINK-18075 > Project: Flink > Issue Type: Improvement > Components: Connectors / Kafka, Formats (JSON, Avro, Parquet, ORC, > SequenceFile) >Affects Versions: 1.11.0, 1.12.0 >Reporter: Seth Wiesman >Assignee: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.11.0 > > > The Kafka consumer and producer do not call the open methods of plain > (De)SerializationSchema interfaces. Only the Keyed and Kafka specific > interfaces. The updated SQL implementations such as > AvroRowDataSeriailzationSchema use these methods and so SQL queries using > avro and kafka will fail in a null pointer exception. > cc [~aljoscha] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-16572) CheckPubSubEmulatorTest is flaky on Azure
[ https://issues.apache.org/jira/browse/FLINK-16572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124611#comment-17124611 ] Robert Metzger commented on FLINK-16572: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=2591=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=1e2bbe5b-4657-50be-1f07-d84bfce5b1f5 > CheckPubSubEmulatorTest is flaky on Azure > - > > Key: FLINK-16572 > URL: https://issues.apache.org/jira/browse/FLINK-16572 > Project: Flink > Issue Type: Bug > Components: Connectors / Google Cloud PubSub, Tests >Affects Versions: 1.11.0 >Reporter: Aljoscha Krettek >Assignee: Richard Deurwaarder >Priority: Critical > Labels: pull-request-available, test-stability > Fix For: 1.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Log: > https://dev.azure.com/aljoschakrettek/Flink/_build/results?buildId=56=logs=1f3ed471-1849-5d3c-a34c-19792af4ad16=ce095137-3e3b-5f73-4b79-c42d3d5f8283=7842 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] godfreyhe commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
godfreyhe commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434317689
##
File path:
flink-table/flink-sql-client/src/test/java/org/apache/flink/table/client/cli/CliClientTest.java
##
@@ -166,6 +167,64 @@ public void write(int b) throws IOException {
}
}
+ @Test
+ public void testUseCatalog() throws Exception {
+ TestingExecutor executor = new TestingExecutorBuilder()
+ .setUseCatalogConsumer((ignored1, catalogName)
-> {
+ if (!catalogName.equals("cat")) {
+ throw new
SqlExecutionException("unexpected catalog name: " + catalogName);
+ }
+ })
+ .build();
+
+ InputStream inputStream = new ByteArrayInputStream("use catalog
cat;\n".getBytes());
+ ByteArrayOutputStream outputStream = new
ByteArrayOutputStream(256);
+ CliClient cliClient = null;
+ SessionContext sessionContext = new
SessionContext("test-session", new Environment());
+ String sessionId = executor.openSession(sessionContext);
+
+ try (Terminal terminal = new DumbTerminal(inputStream,
outputStream)) {
Review comment:
`CliClient` does not inherit from `AutoCloseable`, so we can put
`CliClient` into `try-with-resource` block
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-18068) Job scheduling stops but not exits after throwing non-fatal exception
[
https://issues.apache.org/jira/browse/FLINK-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124610#comment-17124610
]
Jiayi Liao commented on FLINK-18068:
[~azagrebin] Sorry for not checking the JM logs after the period of
{{batchSlotTimeout}}. Your thoughts about the timeout are right, the scheduling
will be triggered again after the period of {{batchSlotTimeout}}.
And yes, I agree with you and [~trohrmann], this should be treated as fatal
error, but I've found other places like {{requestYarnContainerIfRequired}} in
{{onContainersCompleted}}, which have the similar problem. I think we should
fix this in a general way like:
{code:java}
public class FatalErrorRunnable implements Runnable {
private final Runnable runnable;
FatalErrorRunnable(Runnable runnable) {
this.runnable = runnable;
}
@Override
public void run() {
try {
runnable.run();
} catch (Throwable e) {
onFatalError(e);
}
}
}
{code}
> Job scheduling stops but not exits after throwing non-fatal exception
> -
>
> Key: FLINK-18068
> URL: https://issues.apache.org/jira/browse/FLINK-18068
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Affects Versions: 1.10.1
>Reporter: Jiayi Liao
>Priority: Major
>
> The batch job will stop but still be alive with doing nothing for a long time
> (maybe forever?) if any non fatal exception is thrown from interacting with
> YARN. Here is the example :
> {code:java}
> java.lang.IllegalStateException: The RMClient's and YarnResourceManagers
> internal state about the number of pending container requests for resource
> has diverged. Number client's pending container
> requests 40 != Number RM's pending container requests 0.
> at
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:217)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> org.apache.flink.yarn.YarnResourceManager.getPendingRequestsAndCheckConsistency(YarnResourceManager.java:518)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> org.apache.flink.yarn.YarnResourceManager.onContainersOfResourceAllocated(YarnResourceManager.java:431)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> org.apache.flink.yarn.YarnResourceManager.lambda$onContainersAllocated$1(YarnResourceManager.java:395)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:402)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:195)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
> [flink-ljy-1.0.jar:?]
> at
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
> [flink-ljy-1.0.jar:?]
> at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> [flink-ljy-1.0.jar:?]
> at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> [flink-ljy-1.0.jar:?]
> at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.actor.ActorCell.invoke(ActorCell.scala:561)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.dispatch.Mailbox.run(Mailbox.scala:225)
> [flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
> at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
>
[jira] [Updated] (FLINK-17491) Translate Training page on project website
[ https://issues.apache.org/jira/browse/FLINK-17491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-17491: --- Labels: pull-request-available (was: ) > Translate Training page on project website > -- > > Key: FLINK-17491 > URL: https://issues.apache.org/jira/browse/FLINK-17491 > Project: Flink > Issue Type: Improvement > Components: chinese-translation, Project Website >Reporter: David Anderson >Assignee: Li Ying >Priority: Major > Labels: pull-request-available > > Translate the training page for the project website to Chinese. The file is > training.zh.md. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-11909) Provide default failure/timeout/backoff handling strategy for AsyncIO functions
[ https://issues.apache.org/jira/browse/FLINK-11909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124607#comment-17124607 ] alex zhu commented on FLINK-11909: -- I believe most flink users (such as me) are focusing on business or data handling, not a native java programmer。So introducing a built-in retry mechanism would be very great usability improvement。Retry for each record is enough, it can solve "service/database restarting" such common scenarios。 > Provide default failure/timeout/backoff handling strategy for AsyncIO > functions > --- > > Key: FLINK-11909 > URL: https://issues.apache.org/jira/browse/FLINK-11909 > Project: Flink > Issue Type: Improvement > Components: API / DataStream >Reporter: Rong Rong >Assignee: Rong Rong >Priority: Major > > Currently Flink AsyncIO by default fails the entire job when async function > invoke fails [1]. It would be nice to have some default Async IO > failure/timeout handling strategy, or opens up some APIs for AsyncFunction > timeout method to interact with the AsyncWaitOperator. For example (quote > [~suez1224] in [2]): > * FAIL_OPERATOR (default & current behavior) > * FIX_INTERVAL_RETRY (retry with configurable fixed interval up to N times) > * EXP_BACKOFF_RETRY (retry with exponential backoff up to N times) > Discussion also extended to introduce configuration such as: > * MAX_RETRY_COUNT > * RETRY_FAILURE_POLICY > REF: > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/asyncio.html#timeout-handling > [2] > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Backoff-strategies-for-async-IO-functions-tt26580.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] klion26 commented on a change in pull request #12313: [FLINK-17005][docs] Translate the CREATE TABLE ... LIKE syntax documentation to Chinese
klion26 commented on a change in pull request #12313:
URL: https://github.com/apache/flink/pull/12313#discussion_r434312012
##
File path: docs/dev/table/sql/create.zh.md
##
@@ -249,59 +257,60 @@ CREATE TABLE Orders_with_watermark (
);
{% endhighlight %}
-The merging logic of table features can be controlled with `like options`.
+表属性的合并逻辑可以用 `like options` 来控制。
-You can control the merging behavior of:
+可以控制合并的表属性如下：
-* CONSTRAINTS - constraints such as primary and unique keys
-* GENERATED - computed columns
-* OPTIONS - connector options that describe connector and format properties
-* PARTITIONS - partition of the tables
-* WATERMARKS - watermark declarations
+* CONSTRAINTS - 主键和唯一键约束
+* GENERATED - 计算列
+* OPTIONS - 连接器信息、格式化方式等配置项
+* PARTITIONS - 表分区信息
+* WATERMARKS - watermark 定义
-with three different merging strategies:
+并且有三种不同的表属性合并策略：
-* INCLUDING - Includes the feature of the source table, fails on duplicate
entries, e.g. if an option with the same key exists in both tables.
-* EXCLUDING - Does not include the given feature of the source table.
-* OVERWRITING - Includes the feature of the source table, overwrites duplicate
entries of the source table with properties of the new table, e.g. if an option
with the same key exists in both tables, the one from the current statement
will be used.
+* INCLUDING - 新表包含源表（source table）所有的表属性，如果和源表的表属性重复则会直接失败，例如新表和源表存在相同 key 的属性。
+* EXCLUDING - 新表不包含源表指定的任何表属性。
+* OVERWRITING - 新表包含源表的表属性，但如果出现重复项，则会用新表的表属性覆盖源表中的重复表属性，例如，两个表中都存在相同 key
的属性，则会使用当前语句中定义的 key 的属性值。
-Additionally, you can use the `INCLUDING/EXCLUDING ALL` option to specify what
should be the strategy if there was no specific strategy defined, i.e. if you
use `EXCLUDING ALL INCLUDING WATERMARKS` only the watermarks will be included
from the source table.
+并且你可以使用 `INCLUDING/EXCLUDING ALL` 这种声明方式来指定使用怎样的合并策略，例如使用 `EXCLUDING ALL
INCLUDING WATERMARKS`，那么代表只有源表的 WATERMARKS 属性才会被包含进新表。
-Example:
+示例如下：
{% highlight sql %}
--- A source table stored in a filesystem
+
+-- 存储在文件系统的源表
CREATE TABLE Orders_in_file (
user BIGINT,
product STRING,
order_time_string STRING,
order_time AS to_timestamp(order_time)
-
+
Review comment:
这个无关的修改可以去掉
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-web] liying919 opened a new pull request #345: [FLINK-17491] Translate Training page on project website
liying919 opened a new pull request #345: URL: https://github.com/apache/flink-web/pull/345 Translate Training page on project website. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] klion26 commented on pull request #342: [FLINK-17926] Fix the build problem of docker image
klion26 commented on pull request #342: URL: https://github.com/apache/flink-web/pull/342#issuecomment-637952600 Ahh. maybe there is someting not compatible with `ruby:2.5` docker image, I tried to execute the command `docker run --rm --volume="$PWD:/srv/flink-web" --expose=4000 -p 4000:4000 -it ruby:2.5 bash -c 'cd /srv/flink-web && ./build.sh -p'` on my laptop, it failed with the following message, I'll try to find out the reason here. ``` Configuration file: /srv/flink-web/_config.yml bundler: failed to load command: jekyll (/srv/flink-web/.rubydeps/ruby/2.5.0/bin/jekyll) LoadError: libruby-2.5.so.2.5: cannot open shared object file: No such file or directory - /srv/flink-web/.rubydeps/ruby/2.5.0/gems/posix-spawn-0.3.14/lib/posix_spawn_ext.so /srv/flink-web/.rubydeps/ruby/2.5.0/gems/posix-spawn-0.3.14/lib/posix/spawn.rb:2:in `require' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/posix-spawn-0.3.14/lib/posix/spawn.rb:2:in `' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/pygments.rb-0.6.3/lib/pygments/popen.rb:2:in `require' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/pygments.rb-0.6.3/lib/pygments/popen.rb:2:in `' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/pygments.rb-0.6.3/lib/pygments.rb:1:in `require' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/pygments.rb-0.6.3/lib/pygments.rb:1:in `' /srv/flink-web/_plugins/highlightCode.rb:22:in `require' /srv/flink-web/_plugins/highlightCode.rb:22:in `' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/plugin_manager.rb:75:in `require' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/plugin_manager.rb:75:in `block (2 levels) in require_plugin_files' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/plugin_manager.rb:74:in `each' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/plugin_manager.rb:74:in `block in require_plugin_files' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/plugin_manager.rb:73:in `each' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/plugin_manager.rb:73:in `require_plugin_files' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/plugin_manager.rb:18:in `conscientious_require' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/site.rb:97:in `setup' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/site.rb:49:in `initialize' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/commands/build.rb:30:in `new' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/commands/build.rb:30:in `process' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/lib/jekyll/commands/serve.rb:26:in `block (2 levels) in init_with_program' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `block in execute' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `each' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `execute' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/mercenary-0.3.6/lib/mercenary/program.rb:42:in `go' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/mercenary-0.3.6/lib/mercenary.rb:19:in `program' /srv/flink-web/.rubydeps/ruby/2.5.0/gems/jekyll-3.0.5/bin/jekyll:17:in `' /srv/flink-web/.rubydeps/ruby/2.5.0/bin/jekyll:23:in `load' /srv/flink-web/.rubydeps/ruby/2.5.0/bin/jekyll:23:in `' ``` @rmetzger can `./docker/run.sh` success on your side? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12453: [FLINK-18061] [table] TableResult#collect method should return a closeable iterator to avoid resource leak
flinkbot edited a comment on pull request #12453: URL: https://github.com/apache/flink/pull/12453#issuecomment-637941328 ## CI report: * c2f3319900afad80a0f9fac5a8c9aa4cfdadd4ca Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2600) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] klion26 commented on a change in pull request #12237: [FLINK-17290] [chinese-translation, Documentation / Training] Transla…
klion26 commented on a change in pull request #12237:
URL: https://github.com/apache/flink/pull/12237#discussion_r433926976
##
File path: docs/training/streaming_analytics.zh.md
##
@@ -29,123 +29,104 @@ under the License.
## Event Time and Watermarks
-### Introduction
+
+### 概要
-Flink explicitly supports three different notions of time:
+Flink 明确支持以下三种时间语义:
-* _event time:_ the time when an event occurred, as recorded by the device
producing (or storing) the event
+* _事件时间(event time)：_ 事件产生的时间，记录的是设备生产(或者存储)事件的时间
-* _ingestion time:_ a timestamp recorded by Flink at the moment it ingests the
event
+* _摄取时间(ingestion time)：_ Flink 读取事件时记录的时间
-* _processing time:_ the time when a specific operator in your pipeline is
processing the event
+* _处理时间(processing time)：_ Flink pipeline 中具体算子处理事件的时间
-For reproducible results, e.g., when computing the maximum price a stock
reached during the first
-hour of trading on a given day, you should use event time. In this way the
result won't depend on
-when the calculation is performed. This kind of real-time application is
sometimes performed using
-processing time, but then the results are determined by the events that happen
to be processed
-during that hour, rather than the events that occurred then. Computing
analytics based on processing
-time causes inconsistencies, and makes it difficult to re-analyze historic
data or test new
-implementations.
+为了获得可重现的结果，例如在计算过去的特定一天里第一个小时股票的最高价格时，我们应该使用事件时间。这样的话，无论
+什么时间去计算都不会影响输出结果。然而有些人，在实时计算应用中使用处理时间，这样的话，输出结果就会被处理时间点所决
Review comment:
这里 44 行需要和 43 行合并，否则 “无论” 和 “什么” 之间会有空格
45 和 44 也需要合并，否则 “所决” 和 “定” 之间有空格
`This kind of real-time application is sometimes performed using processing
time, but then the results are determined by the events that happen to be
processed during that hour, rather than the events that occurred then.`
这句话的翻译直接翻译即可，比如 `然后使用 processing time
的那些实时应用程序的处理结果，则由那一小时中所处理的数据所决定`（不一定是这个翻译，可以用你自己的语句描述）
##
File path: docs/training/streaming_analytics.zh.md
##
@@ -29,123 +29,104 @@ under the License.
## Event Time and Watermarks
-### Introduction
+
+### 概要
-Flink explicitly supports three different notions of time:
+Flink 明确支持以下三种时间语义:
-* _event time:_ the time when an event occurred, as recorded by the device
producing (or storing) the event
+* _事件时间(event time)：_ 事件产生的时间，记录的是设备生产(或者存储)事件的时间
-* _ingestion time:_ a timestamp recorded by Flink at the moment it ingests the
event
+* _摄取时间(ingestion time)：_ Flink 读取事件时记录的时间
-* _processing time:_ the time when a specific operator in your pipeline is
processing the event
+* _处理时间(processing time)：_ Flink pipeline 中具体算子处理事件的时间
-For reproducible results, e.g., when computing the maximum price a stock
reached during the first
-hour of trading on a given day, you should use event time. In this way the
result won't depend on
-when the calculation is performed. This kind of real-time application is
sometimes performed using
-processing time, but then the results are determined by the events that happen
to be processed
-during that hour, rather than the events that occurred then. Computing
analytics based on processing
-time causes inconsistencies, and makes it difficult to re-analyze historic
data or test new
-implementations.
+为了获得可重现的结果，例如在计算过去的特定一天里第一个小时股票的最高价格时，我们应该使用事件时间。这样的话，无论
+什么时间去计算都不会影响输出结果。然而有些人，在实时计算应用中使用处理时间，这样的话，输出结果就会被处理时间点所决
+定，而不是生产事件的时间。基于处理时间会导致多次计算的结果不一致，也可能会导致再次分析历史数据或者测试新代码变得异常困难。
-### Working with Event Time
+
+### 使用 Event Time
-By default, Flink will use processing time. To change this, you can set the
Time Characteristic:
+Flink 在默认情况下是使用处理时间。也可以通过下面配置来告诉 Flink 选择哪种时间语义:
{% highlight java %}
final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
{% endhighlight %}
-If you want to use event time, you will also need to supply a Timestamp
Extractor and Watermark
-Generator that Flink will use to track the progress of event time. This will
be covered in the
-section below on [Working with Watermarks]({% link
-training/streaming_analytics.zh.md %}#working-with-watermarks), but first we
should explain what
-watermarks are.
+如果想要使用事件时间，需要额外给 Flink 提供一个时间戳的提取器和 Watermark 生成器，Flink 将使用它们来跟踪事件时间的进度。这
+将在选节[使用 Watermarks]({% link training/streaming_analytics.zh.md
%}#Working-with-Watermarks)中介绍，但是首先我们需要解释一下
+ watermarks 是什么。
### Watermarks
-Let's work through a simple example that will show why watermarks are needed,
and how they work.
+让我们通过一个简单的示例来演示为什么需要 watermarks 及其工作方式。
-In this example you have a stream of timestamped events that arrive somewhat
out of order, as shown
-below. The numbers shown are timestamps that indicate when these events
actually occurred. The first
-event to arrive happened at time 4, and it is followed by an event that
happened earlier, at time 2,
-and so on:
[jira] [Updated] (FLINK-17686) Add document to dataGen, print, blackhole connectors
[ https://issues.apache.org/jira/browse/FLINK-17686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17686: --- Priority: Critical (was: Major) > Add document to dataGen, print, blackhole connectors > > > Key: FLINK-17686 > URL: https://issues.apache.org/jira/browse/FLINK-17686 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Affects Versions: 1.11.0 >Reporter: Jingsong Lee >Assignee: Shengkai Fang >Priority: Critical > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17831) Add documentation for the new Kafka connector
[ https://issues.apache.org/jira/browse/FLINK-17831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17831: --- Priority: Critical (was: Major) > Add documentation for the new Kafka connector > - > > Key: FLINK-17831 > URL: https://issues.apache.org/jira/browse/FLINK-17831 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Jark Wu >Assignee: Danny Chen >Priority: Critical > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17776) Add documentation for DDL in hive dialect
[ https://issues.apache.org/jira/browse/FLINK-17776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17776: --- Priority: Critical (was: Major) > Add documentation for DDL in hive dialect > - > > Key: FLINK-17776 > URL: https://issues.apache.org/jira/browse/FLINK-17776 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Jingsong Lee >Assignee: Rui Li >Priority: Critical > Labels: pull-request-available > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #12453: [FLINK-18061] [table] TableResult#collect method should return a closeable iterator to avoid resource leak
flinkbot commented on pull request #12453: URL: https://github.com/apache/flink/pull/12453#issuecomment-637941328 ## CI report: * c2f3319900afad80a0f9fac5a8c9aa4cfdadd4ca UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12422: [FLINK-17901][python] Add module interface in PyFlink
flinkbot edited a comment on pull request #12422: URL: https://github.com/apache/flink/pull/12422#issuecomment-636647115 ## CI report: * 3a387da20f29d9b1f6baf1e57c6d9979d27dd81f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2509) * a57beb047e52bacd60d9c19cbcd31bc9b1d88ec4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2599) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] libenchao commented on a change in pull request #12436: [FLINK-17847][table sql / planner] ArrayIndexOutOfBoundsException happens in StreamExecCalc operator
libenchao commented on a change in pull request #12436:
URL: https://github.com/apache/flink/pull/12436#discussion_r434297494
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala
##
@@ -1643,19 +1644,35 @@ object ScalarOperatorGens {
val resultTypeTerm = primitiveTypeTermForType(componentInfo)
val defaultTerm = primitiveDefaultValue(componentInfo)
+if (index.literalValue.isDefined &&
+index.literalValue.get.isInstanceOf[Int] &&
+index.literalValue.get.asInstanceOf[Int] < 1) {
+ throw new ValidationException(s"Array element access needs an index
starting at 1 but was " +
+s"${index.literalValue.get.asInstanceOf[Int]}.")
+}
val idxStr = s"${index.resultTerm} - 1"
val arrayIsNull = s"${array.resultTerm}.isNullAt($idxStr)"
val arrayGet =
rowFieldReadAccess(ctx, idxStr, array.resultTerm, componentInfo)
+/**
+ * Return null when array index out of bounds which follows Calcite's
behaviour.
+ * @see org.apache.calcite.sql.fun.SqlStdOperatorTable
+ */
val arrayAccessCode =
- s"""
- |${array.code}
- |${index.code}
- |boolean $nullTerm = ${array.nullTerm} || ${index.nullTerm} ||
$arrayIsNull;
- |$resultTypeTerm $resultTerm = $nullTerm ? $defaultTerm : $arrayGet;
- |""".stripMargin
-
+s"""
+|${array.code}
+|${index.code}
+|$resultTypeTerm $resultTerm;
+|boolean $nullTerm;
+|if (${idxStr} < 0 || ${idxStr} >= ${array.resultTerm}.size()) {
Review comment:
What I want to express is, the index could be a literal or a field
reference.
And we want to throw Exception for index < 1, there are two cases that index
< 1:
- literal value, it has been checked in code gen, and throws
ValidationException.
- field reference, currently you just return null for this case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-16975) Add docs for FileSystem connector
[ https://issues.apache.org/jira/browse/FLINK-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-16975: --- Priority: Critical (was: Major) > Add docs for FileSystem connector > - > > Key: FLINK-16975 > URL: https://issues.apache.org/jira/browse/FLINK-16975 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Affects Versions: 1.11.0 >Reporter: Leonard Xu >Assignee: Jingsong Lee >Priority: Critical > Labels: pull-request-available > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17599) Update documents due to FLIP-84
[ https://issues.apache.org/jira/browse/FLINK-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17599: --- Priority: Critical (was: Major) > Update documents due to FLIP-84 > --- > > Key: FLINK-17599 > URL: https://issues.apache.org/jira/browse/FLINK-17599 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Kurt Young >Assignee: godfrey he >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17832) Add documentation for the new Elasticsearch connector
[ https://issues.apache.org/jira/browse/FLINK-17832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17832: --- Priority: Critical (was: Major) > Add documentation for the new Elasticsearch connector > - > > Key: FLINK-17832 > URL: https://issues.apache.org/jira/browse/FLINK-17832 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Jark Wu >Assignee: Shengkai Fang >Priority: Critical > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-18065) Add documentation for new scalar/table functions
[ https://issues.apache.org/jira/browse/FLINK-18065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-18065: --- Priority: Critical (was: Major) > Add documentation for new scalar/table functions > > > Key: FLINK-18065 > URL: https://issues.apache.org/jira/browse/FLINK-18065 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Timo Walther >Assignee: Timo Walther >Priority: Critical > > Write documentation for scalar/table functions of FLIP-65. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17829) Add documentation for the new JDBC connector
[ https://issues.apache.org/jira/browse/FLINK-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17829: --- Priority: Critical (was: Major) > Add documentation for the new JDBC connector > > > Key: FLINK-17829 > URL: https://issues.apache.org/jira/browse/FLINK-17829 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Jark Wu >Assignee: Leonard Xu >Priority: Critical > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-18066) Add documentation for how to develop a new table source/sink
[ https://issues.apache.org/jira/browse/FLINK-18066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-18066: --- Priority: Critical (was: Major) > Add documentation for how to develop a new table source/sink > > > Key: FLINK-18066 > URL: https://issues.apache.org/jira/browse/FLINK-18066 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Timo Walther >Assignee: Timo Walther >Priority: Critical > > Covers how to write a custom source/sink and format using FLIP-95 interfaces. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17406) Add documentation about dynamic table options
[ https://issues.apache.org/jira/browse/FLINK-17406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17406: --- Priority: Critical (was: Major) > Add documentation about dynamic table options > - > > Key: FLINK-17406 > URL: https://issues.apache.org/jira/browse/FLINK-17406 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Kurt Young >Assignee: Danny Chen >Priority: Critical > Labels: pull-request-available > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17830) Add documentation for the new HBase connector
[ https://issues.apache.org/jira/browse/FLINK-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17830: --- Priority: Critical (was: Major) > Add documentation for the new HBase connector > - > > Key: FLINK-17830 > URL: https://issues.apache.org/jira/browse/FLINK-17830 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Jark Wu >Assignee: Jark Wu >Priority: Critical > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17931) Document fromValues clause
[ https://issues.apache.org/jira/browse/FLINK-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17931: --- Priority: Critical (was: Major) > Document fromValues clause > -- > > Key: FLINK-17931 > URL: https://issues.apache.org/jira/browse/FLINK-17931 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Table SQL / API >Reporter: Dawid Wysakowicz >Assignee: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17834) Add document for Hive streaming source
[ https://issues.apache.org/jira/browse/FLINK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17834: --- Priority: Critical (was: Major) > Add document for Hive streaming source > -- > > Key: FLINK-17834 > URL: https://issues.apache.org/jira/browse/FLINK-17834 > Project: Flink > Issue Type: Sub-task >Reporter: Danny Chen >Assignee: Jingsong Lee >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17836) Add document for Hive dim join
[ https://issues.apache.org/jira/browse/FLINK-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17836: --- Priority: Critical (was: Major) > Add document for Hive dim join > -- > > Key: FLINK-17836 > URL: https://issues.apache.org/jira/browse/FLINK-17836 > Project: Flink > Issue Type: Sub-task >Reporter: Danny Chen >Assignee: Rui Li >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17835) Add document for Hive streaming sink
[ https://issues.apache.org/jira/browse/FLINK-17835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated FLINK-17835: --- Priority: Critical (was: Major) > Add document for Hive streaming sink > > > Key: FLINK-17835 > URL: https://issues.apache.org/jira/browse/FLINK-17835 > Project: Flink > Issue Type: Sub-task >Reporter: Danny Chen >Assignee: Jingsong Lee >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] Jiayi-Liao edited a comment on pull request #12444: [FLINK-18067]Change default value of yarnMinAllocationMB from zero to DEFAULT_RM_S…
Jiayi-Liao edited a comment on pull request #12444: URL: https://github.com/apache/flink/pull/12444#issuecomment-637920335 @wangyang0918 Thanks for your review. I've addressed your comments and also update the description(the CI failure is caused by checkstyle error which is already fixed in the latest commit). Please take another look when you can spare time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] leonardBang commented on a change in pull request #12436: [FLINK-17847][table sql / planner] ArrayIndexOutOfBoundsException happens in StreamExecCalc operator
leonardBang commented on a change in pull request #12436:
URL: https://github.com/apache/flink/pull/12436#discussion_r434295803
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala
##
@@ -1643,19 +1644,35 @@ object ScalarOperatorGens {
val resultTypeTerm = primitiveTypeTermForType(componentInfo)
val defaultTerm = primitiveDefaultValue(componentInfo)
+if (index.literalValue.isDefined &&
+index.literalValue.get.isInstanceOf[Int] &&
+index.literalValue.get.asInstanceOf[Int] < 1) {
+ throw new ValidationException(s"Array element access needs an index
starting at 1 but was " +
+s"${index.literalValue.get.asInstanceOf[Int]}.")
+}
val idxStr = s"${index.resultTerm} - 1"
val arrayIsNull = s"${array.resultTerm}.isNullAt($idxStr)"
val arrayGet =
rowFieldReadAccess(ctx, idxStr, array.resultTerm, componentInfo)
+/**
+ * Return null when array index out of bounds which follows Calcite's
behaviour.
+ * @see org.apache.calcite.sql.fun.SqlStdOperatorTable
+ */
val arrayAccessCode =
- s"""
- |${array.code}
- |${index.code}
- |boolean $nullTerm = ${array.nullTerm} || ${index.nullTerm} ||
$arrayIsNull;
- |$resultTypeTerm $resultTerm = $nullTerm ? $defaultTerm : $arrayGet;
- |""".stripMargin
-
+s"""
+|${array.code}
+|${index.code}
+|$resultTypeTerm $resultTerm;
+|boolean $nullTerm;
+|if (${idxStr} < 0 || ${idxStr} >= ${array.resultTerm}.size()) {
Review comment:
> consider this case:
>
> ```sql
> CREATE TABLE my_source (
> idx int,
> my_arr array
> ) WITH (...);
>
> SELECT my_arr[idx] FROM my_source;
> ```
For this case, idx has been an integer value rather than an input reference
in runtime IIUC.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-18049) The Flink kafka consumer job will be interrupted if the upstream kafka producer change the AVRO schema
[
https://issues.apache.org/jira/browse/FLINK-18049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124584#comment-17124584
]
Jark Wu commented on FLINK-18049:
-
Thanks for the discussion and looking into this problem [~openinx]. If I
understand correctly, what you want to fix is such case:
0. there is a Kafka Avro topic whose schema is [int a, string b].
1. there is a long running Flink SQL job which consumes Kafka Avro data using
DDL {{CREATE TABLE xxx (a INT, b STRING) WITH (... format=avro)}}.
2. the producer of this Kafka Avro topic evolves their schema with
backward-compatible by adding a column which default null. [int a, string b,
double c default null]. In theory, the existing consumers of this topic should
work fine, according to the [Avro schema
rule|http://avro.apache.org/docs/current/spec.html#Schema+Resolution].
3. However, there is an exception thrown in the running Flink SQL job.
If this is the case (and I think the added test is verifying this case), I
think it's reasonable to improve the current implementation.
Besides, in another case, if we are using DDL {{CREATE TABLE xxx (a INT, b
STRING) WITH (... format=avro)}} to read existing Kafka topic with avro schema
[int a, string b, double c default null], an error will happen too. So I think
this is a bug.
> The Flink kafka consumer job will be interrupted if the upstream kafka
> producer change the AVRO schema
> --
>
> Key: FLINK-18049
> URL: https://issues.apache.org/jira/browse/FLINK-18049
> Project: Flink
> Issue Type: Bug
>Reporter: Zheng Hu
>Priority: Critical
> Labels: pull-request-available
>
> We have encountered a critical case from online services. we have the data
> pipeline: (producer) -> (kafka) -> (flink consumer job), and all those
> records are encoded in AVRO format. Once the producer changed the AVRO
> schema , says adding an extra column to the existing schema and writing few
> data into the Kafka.
> Then the downstream flink job crashed with the following stacktrace:
> {code}
> ==WARNING== allocating large
> array--thread_id[0x7fccd9c16800]--thread_name[Source: Custom Source
> (1/1)]--array_size[1590681120 bytes]--array_length[1590681103 elememts]
> os_prio=0 tid=0x7fccd9c16800 nid=0x226c0 runnable
> at org.shaded.apache.avro.util.Utf8.setByteLength(Utf8.java:78)
> at
> org.shaded.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:261)
> at
> org.shaded.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:272)
> at
> org.shaded.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:214)
> at
> org.shaded.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:412)
> at
> org.shaded.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:181)
> at
> org.shaded.apache.avro.specific.SpecificDatumReader.readField(SpecificDatumReader.java:116)
> at
> org.shaded.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222)
> at
> org.shaded.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
> at
> org.shaded.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
> at
> org.shaded.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
> at
> org.apache.flink.formats.avro.AvroRowDeserializationSchema.deserialize(AvroRowDeserializationSchema.java:167)
> at
> org.apache.flink.formats.avro.AvroRowDeserializationSchema.deserialize(AvroRowDeserializationSchema.java:78)
> at
> org.apache.flink.streaming.util.serialization.KeyedDeserializationSchemaWrapper.deserialize(KeyedDeserializationSchemaWrapper.java:44)
> at
> org.apache.flink.streaming.connectors.kafka.internal.Kafka09Fetcher.runFetchLoop(Kafka09Fetcher.java:192)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:771)
> at
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:120)
> at
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:74)
> at
> org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:129)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:398)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:736)
> at java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] dianfu commented on a change in pull request #12422: [FLINK-17901][python] Add module interface in PyFlink
dianfu commented on a change in pull request #12422: URL: https://github.com/apache/flink/pull/12422#discussion_r434294304 ## File path: flink-python/pyflink/table/module.py ## @@ -0,0 +1,45 @@ + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from pyflink.java_gateway import get_gateway + +__all__ = ['Module', 'HiveModule'] + + +class Module(object): +""" +Modules define a set of metadata, including functions, user defined types, operators, rules, +etc. +Metadata from modules are regarded as built-in or system metadata that users can take advantages +of. +""" + +def __init__(self, j_module): +self._j_module = j_module + + +class HiveModule(Module): +""" +Module to provide Hive built-in metadata. +""" + +def __init__(self, j_hive_module=None): +gateway = get_gateway() + +if j_hive_module is None: Review comment: it seems that the parameter `j_hive_module` is not necessary. I noticed that there is such a parameter in HiveCatalog, however I guess we should also remove it. What do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #12453: [FLINK-18061] [table] TableResult#collect method should return a closeable iterator to avoid resource leak
flinkbot commented on pull request #12453: URL: https://github.com/apache/flink/pull/12453#issuecomment-637937506 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c2f3319900afad80a0f9fac5a8c9aa4cfdadd4ca (Wed Jun 03 03:57:13 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-18061).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-18080) Translate the "Kerberos Authentication Setup and Configuration" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124578#comment-17124578 ] Yangze Guo commented on FLINK-18080: Could someone kindly assign this to me? > Translate the "Kerberos Authentication Setup and Configuration" page into > Chinese > - > > Key: FLINK-18080 > URL: https://issues.apache.org/jira/browse/FLINK-18080 > Project: Flink > Issue Type: Sub-task >Reporter: Yangze Guo >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-18080) Translate the "Kerberos Authentication Setup and Configuration" page into Chinese
Yangze Guo created FLINK-18080: -- Summary: Translate the "Kerberos Authentication Setup and Configuration" page into Chinese Key: FLINK-18080 URL: https://issues.apache.org/jira/browse/FLINK-18080 Project: Flink Issue Type: Sub-task Reporter: Yangze Guo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-18061) TableResult#collect should return closeable iterator to avoid resource leak
[
https://issues.apache.org/jira/browse/FLINK-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-18061:
---
Labels: pull-request-available (was: )
> TableResult#collect should return closeable iterator to avoid resource leak
> ---
>
> Key: FLINK-18061
> URL: https://issues.apache.org/jira/browse/FLINK-18061
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Reporter: godfrey he
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> as discussed in ML:
> http://mail-archives.apache.org/mod_mbox/flink-dev/202005.mbox/%3cd4ee47e1-0214-aa2f-f5ac-c9daf708e...@apache.org%3e,
> we should return a closeable iterator for TableResult#collect method *to
> avoid resource leak*. The suggested change is:
> {code:java}
> public interface TableResult {
> CloseableIterator collect();
> }
> {code}
> we use existing {{org.apache.flink.util.CloseableIterator}} instead of
> additional api class, and {{CollectResultIterator}} can also inherit from
> {{CloseableIterator}}.
> This change does not break current api.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-18072) HBaseLookupFunction can not work with new internal data structure RowData
[ https://issues.apache.org/jira/browse/FLINK-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-18072: --- Assignee: Leonard Xu > HBaseLookupFunction can not work with new internal data structure RowData > - > > Key: FLINK-18072 > URL: https://issues.apache.org/jira/browse/FLINK-18072 > Project: Flink > Issue Type: Bug > Components: Connectors / HBase >Affects Versions: 1.11.0 >Reporter: Leonard Xu >Assignee: Leonard Xu >Priority: Major > Fix For: 1.11.0, 1.12.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] godfreyhe opened a new pull request #12453: [FLINK-18061] [table] TableResult#collect method should return a closeable iterator to avoid resource leak
godfreyhe opened a new pull request #12453: URL: https://github.com/apache/flink/pull/12453 ## What is the purpose of the change *as discussed in ML: http://mail-archives.apache.org/mod_mbox/flink-dev/202005.mbox/%3cd4ee47e1-0214-aa2f-f5ac-c9daf708e...@apache.org%3e, we should return a closeable iterator for TableResult#collect method to avoid resource leak. This pr aims to change the return type of TableResult#collect method from Iterator to CloseableIterator* ## Brief change log - *change the return type of TableResult#collect method from Iterator to CloseableIterator* - *CollectResultIterator inherits from CloseableIterator.* ## Verifying this change *(Please pick either of the following options)* This change added tests and can be verified as follows: - *Extended TableITCase to verify close logic* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**yes** / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented**) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12452: [FLINK-18056][fs-connector] Removing the remaining in-progress file on initialization for hadoop path-based writer
flinkbot edited a comment on pull request #12452: URL: https://github.com/apache/flink/pull/12452#issuecomment-637922636 ## CI report: * 7cb10bd1269179bbe66f5b840c0f0d2494c66a55 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2598) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12422: [FLINK-17901][python] Add module interface in PyFlink
flinkbot edited a comment on pull request #12422: URL: https://github.com/apache/flink/pull/12422#issuecomment-636647115 ## CI report: * 3a387da20f29d9b1f6baf1e57c6d9979d27dd81f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2509) * a57beb047e52bacd60d9c19cbcd31bc9b1d88ec4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-13866) develop testing plan for many Hive versions that we support
[ https://issues.apache.org/jira/browse/FLINK-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124577#comment-17124577 ] Rui Li commented on FLINK-13866: We already have the maven profiles to test different hive versions. I think we can close this one, if nobody objects. > develop testing plan for many Hive versions that we support > --- > > Key: FLINK-13866 > URL: https://issues.apache.org/jira/browse/FLINK-13866 > Project: Flink > Issue Type: Test > Components: Connectors / Hive >Reporter: Bowen Li >Assignee: Xuefu Zhang >Priority: Major > > with FLINK-13841, we will start to support quite a few hive versions, let > alone other major versions like 1.1, 2.2, and 3.x. > We need to come up with a testing plan to cover all these Hive versions to > guarantee 1) help identify and fix breaking changes ASAP, 2) minimize > developers' efforts in manually test and maintain compatibilities of all > these Hive versions, and automate as much as possible. > Set it to 1.10.0 for now. > cc [~xuefuz] [~lirui] [~Terry1897] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] jxeditor commented on a change in pull request #12431: [FLINK-18055] [sql-client] Fix catalog/database does not exist in sql client
jxeditor commented on a change in pull request #12431:
URL: https://github.com/apache/flink/pull/12431#discussion_r434289783
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
Review comment:
```suggestion
operands = new
String[]{StringUtils.strip(((UseCatalogOperation) operation).getCatalogName(),
"`")};
```
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/SqlCommandParser.java
##
@@ -132,11 +132,10 @@ private SqlCommandParser() {
cmd = SqlCommand.DROP_CATALOG;
} else if (operation instanceof UseCatalogOperation) {
cmd = SqlCommand.USE_CATALOG;
- operands = new String[] { String.format("`%s`",
((UseCatalogOperation) operation).getCatalogName()) };
+ operands = new String[] { ((UseCatalogOperation)
operation).getCatalogName() };
} else if (operation instanceof UseDatabaseOperation) {
cmd = SqlCommand.USE;
- UseDatabaseOperation op = ((UseDatabaseOperation)
operation);
- operands = new String[] { String.format("`%s`.`%s`",
op.getCatalogName(), op.getDatabaseName()) };
+ operands = new String[] { ((UseDatabaseOperation)
operation).getDatabaseName() };
Review comment:
```suggestion
operands = new String[]{String.format("%s.%s",
StringUtils.strip(op.getCatalogName(), "`"),
StringUtils.strip(op.getDatabaseName(), "`"))};
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-17733) Add documentation for real-time hive
[ https://issues.apache.org/jira/browse/FLINK-17733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-17733: --- Priority: Blocker (was: Major) > Add documentation for real-time hive > > > Key: FLINK-17733 > URL: https://issues.apache.org/jira/browse/FLINK-17733 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Jingsong Lee >Priority: Blocker > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-18061) TableResult#collect should return closeable iterator to avoid resource leak
[
https://issues.apache.org/jira/browse/FLINK-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated FLINK-18061:
---
Description:
as discussed in ML:
http://mail-archives.apache.org/mod_mbox/flink-dev/202005.mbox/%3cd4ee47e1-0214-aa2f-f5ac-c9daf708e...@apache.org%3e,
we should return a closeable iterator for TableResult#collect method *to avoid
resource leak*. The suggested change is:
{code:java}
public interface TableResult {
CloseableIterator collect();
}
{code}
we use existing {{org.apache.flink.util.CloseableIterator}} instead of
additional api class, and {{CollectResultIterator}} can also inherit from
{{CloseableIterator}}.
This change does not break current api.
was:
as discussed in ML:
http://mail-archives.apache.org/mod_mbox/flink-dev/202005.mbox/%3cd4ee47e1-0214-aa2f-f5ac-c9daf708e...@apache.org%3e,
we should return a closeable iterator for TableResult#collect method *to avoid
resource leak*. The suggested change is:
{code:java}
public interface TableResult {
CloseableRowIterator collect();
}
public interface CloseableRowIterator extends Iterator, AutoCloseable {
}
{code}
This change does not break current api.
> TableResult#collect should return closeable iterator to avoid resource leak
> ---
>
> Key: FLINK-18061
> URL: https://issues.apache.org/jira/browse/FLINK-18061
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Reporter: godfrey he
>Priority: Blocker
> Fix For: 1.11.0
>
>
> as discussed in ML:
> http://mail-archives.apache.org/mod_mbox/flink-dev/202005.mbox/%3cd4ee47e1-0214-aa2f-f5ac-c9daf708e...@apache.org%3e,
> we should return a closeable iterator for TableResult#collect method *to
> avoid resource leak*. The suggested change is:
> {code:java}
> public interface TableResult {
> CloseableIterator collect();
> }
> {code}
> we use existing {{org.apache.flink.util.CloseableIterator}} instead of
> additional api class, and {{CollectResultIterator}} can also inherit from
> {{CloseableIterator}}.
> This change does not break current api.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15261) add dedicated documentation for blink planner
[ https://issues.apache.org/jira/browse/FLINK-15261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-15261: --- Fix Version/s: (was: 1.10.2) (was: 1.11.0) > add dedicated documentation for blink planner > -- > > Key: FLINK-15261 > URL: https://issues.apache.org/jira/browse/FLINK-15261 > Project: Flink > Issue Type: Task > Components: Documentation, Table SQL / Planner >Affects Versions: 1.10.0 >Reporter: Bowen Li >Assignee: Kurt Young >Priority: Major > > we are missing a dedicated page under `Table API and SQL` section to describe > in detail what are the advantages of blink planner, and why users should use > it over the legacy one. > I'm trying to reference a blink planner page in Flink's Hive documentation, > and realized there's even not one yet -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15242) Add doc to introduce ddls or dmls supported by sql cli
[ https://issues.apache.org/jira/browse/FLINK-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-15242: --- Priority: Critical (was: Major) > Add doc to introduce ddls or dmls supported by sql cli > -- > > Key: FLINK-15242 > URL: https://issues.apache.org/jira/browse/FLINK-15242 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Table SQL / Client >Affects Versions: 1.10.0 >Reporter: Terry Wang >Priority: Critical > Fix For: 1.11.0, 1.10.2 > > > Now in the document of sql client > https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sqlClient.html, > there isn't a part to introduce the ddls/dmls in a whole story. We should > complete it before the 1.10 release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17908) Vague document about Kafka config in SQL-CLI
[ https://issues.apache.org/jira/browse/FLINK-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-17908: --- Priority: Critical (was: Minor) > Vague document about Kafka config in SQL-CLI > > > Key: FLINK-17908 > URL: https://issues.apache.org/jira/browse/FLINK-17908 > Project: Flink > Issue Type: Improvement > Components: Documentation, Table SQL / API >Affects Versions: 1.11.0 >Reporter: Shengkai Fang >Priority: Critical > Fix For: 1.11.0 > > > Currently Flink doesn't offer any default config value for Kafka and use the > deault config from Kafka. However, it uses the different config value when > describe how to use Kafka Connector in sql-client. Document of the connector > use value 'ealiest-offset' for 'connector.startup-mode', which is different > from Kafka's default behaviour. I think this vague document may mislead > users, especially for newbies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14256) [Umbrella] Introduce FileSystemTableFactory with partitioned support
[ https://issues.apache.org/jira/browse/FLINK-14256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-14256: --- Summary: [Umbrella] Introduce FileSystemTableFactory with partitioned support (was: Introduce FileSystemTableFactory with partitioned support) > [Umbrella] Introduce FileSystemTableFactory with partitioned support > > > Key: FLINK-14256 > URL: https://issues.apache.org/jira/browse/FLINK-14256 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Planner >Reporter: Jingsong Lee >Assignee: Jingsong Lee >Priority: Major > Fix For: 1.11.0 > > > Introduce FileSystemTableFactory to unify all file system connectors. > More information in > [https://cwiki.apache.org/confluence/display/FLINK/FLIP-115%3A+Filesystem+connector+in+Table] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17635) Add documentation about view support
[ https://issues.apache.org/jira/browse/FLINK-17635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-17635: --- Priority: Blocker (was: Major) > Add documentation about view support > - > > Key: FLINK-17635 > URL: https://issues.apache.org/jira/browse/FLINK-17635 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: Kurt Young >Assignee: Caizhi Weng >Priority: Blocker > Fix For: 1.11.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-13866) develop testing plan for many Hive versions that we support
[ https://issues.apache.org/jira/browse/FLINK-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-13866: --- Fix Version/s: (was: 1.11.0) > develop testing plan for many Hive versions that we support > --- > > Key: FLINK-13866 > URL: https://issues.apache.org/jira/browse/FLINK-13866 > Project: Flink > Issue Type: Test > Components: Connectors / Hive >Reporter: Bowen Li >Assignee: Xuefu Zhang >Priority: Major > > with FLINK-13841, we will start to support quite a few hive versions, let > alone other major versions like 1.1, 2.2, and 3.x. > We need to come up with a testing plan to cover all these Hive versions to > guarantee 1) help identify and fix breaking changes ASAP, 2) minimize > developers' efforts in manually test and maintain compatibilities of all > these Hive versions, and automate as much as possible. > Set it to 1.10.0 for now. > cc [~xuefuz] [~lirui] [~Terry1897] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-16448) add documentation for Hive table source and sink parallelism setting strategy
[ https://issues.apache.org/jira/browse/FLINK-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-16448: --- Component/s: Documentation > add documentation for Hive table source and sink parallelism setting strategy > - > > Key: FLINK-16448 > URL: https://issues.apache.org/jira/browse/FLINK-16448 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive, Documentation >Reporter: Bowen Li >Assignee: Jingsong Lee >Priority: Major > Fix For: 1.11.0 > > > per user-zh mailing list question, would be beneficial to add documentation > for Hive table sink parallelism setting strategy -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-16448) add documentation for Hive table source and sink parallelism setting strategy
[ https://issues.apache.org/jira/browse/FLINK-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-16448: --- Priority: Critical (was: Major) > add documentation for Hive table source and sink parallelism setting strategy > - > > Key: FLINK-16448 > URL: https://issues.apache.org/jira/browse/FLINK-16448 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive, Documentation >Reporter: Bowen Li >Assignee: Jingsong Lee >Priority: Critical > Fix For: 1.11.0 > > > per user-zh mailing list question, would be beneficial to add documentation > for Hive table sink parallelism setting strategy -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-18079) KafkaShuffle Manual Tests
[ https://issues.apache.org/jira/browse/FLINK-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-18079: - Description: Manual Tests and Results to demonstrate KafkaShuffle working as expected. (was: Manual Tests and Results to demonstrate KafkaShuffle is working as expected.) > KafkaShuffle Manual Tests > - > > Key: FLINK-18079 > URL: https://issues.apache.org/jira/browse/FLINK-18079 > Project: Flink > Issue Type: Improvement > Components: API / DataStream, Runtime / Checkpointing >Affects Versions: 1.11.0 >Reporter: Yuan Mei >Priority: Major > Fix For: 1.11.0 > > > Manual Tests and Results to demonstrate KafkaShuffle working as expected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #12452: [FLINK-18056][fs-connector] Removing the remaining in-progress file on initialization for hadoop path-based writer
flinkbot edited a comment on pull request #12452: URL: https://github.com/apache/flink/pull/12452#issuecomment-637922636 ## CI report: * 7cb10bd1269179bbe66f5b840c0f0d2494c66a55 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2598) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12451: [FLINK-17857][e2e] Fix Kubernetes and docker e2e tests on Mac OS
flinkbot edited a comment on pull request #12451: URL: https://github.com/apache/flink/pull/12451#issuecomment-637922585 ## CI report: * 4c93b68fa64dd913f53d69dad4eab467e78e6507 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2597) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12444: [FLINK-18067]Change default value of yarnMinAllocationMB from zero to DEFAULT_RM_S…
flinkbot edited a comment on pull request #12444: URL: https://github.com/apache/flink/pull/12444#issuecomment-637596755 ## CI report: * 16ec8b694dc07564b09b60040d20b0da9c0e80c4 UNKNOWN * 59c44869c548ce9b1930d5b742a6f2c256c5a5b5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2576) * c23450c9e7735ca7bad0ff97c37480de6c420d0e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2596) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-18079) KafkaShuffle Manual Tests
Yuan Mei created FLINK-18079: Summary: KafkaShuffle Manual Tests Key: FLINK-18079 URL: https://issues.apache.org/jira/browse/FLINK-18079 Project: Flink Issue Type: Improvement Components: API / DataStream, Runtime / Checkpointing Affects Versions: 1.11.0 Reporter: Yuan Mei Fix For: 1.11.0 Manual Tests and Results to demonstrate KafkaShuffle is working as expected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] danny0405 commented on a change in pull request #12386: [FLINK-17995][docs][connectors] Redesign Table & SQL Connectors page and add HBase connector documentation
danny0405 commented on a change in pull request #12386:
URL: https://github.com/apache/flink/pull/12386#discussion_r434281449
##
File path: docs/dev/table/connectors/index.zh.md
##
@@ -0,0 +1,268 @@
+---
+title: "Table & SQL Connectors"
+nav-id: sql-connectors
+nav-parent_id: connectors-root
+nav-pos: 2
+nav-show_overview: true
+---
+
+
+
+Flink's Table API & SQL programs can be connected to other external systems
for reading and writing both batch and streaming tables. A table source
provides access to data which is stored in external systems (such as a
database, key-value store, message queue, or file system). A table sink emits a
table to an external storage system. Depending on the type of source and sink,
they support different formats such as CSV, Avro, Parquet, or ORC.
+
+This page describes how to register table sources and table sinks in Flink
using the natively supported connectors. After a source or sink has been
registered, it can be accessed by Table API & SQL statements.
+
+NOTE If you want to implement your own
*custom* table source or sink, have a look at the [user-defined sources & sinks
page](sourceSinks.html).
+
+Attention Flink Table & SQL introduces
a new set of connector options since 1.11.0, if you are using the legacy
connector options, please refer to the [legacy documentation]({{ site.baseurl
}}/dev/table/connect.html).
+
+* This will be replaced by the TOC
+{:toc}
+
+Supported Connectors
+
+
+Flink natively support various connectors. The following tables list all
available connectors.
+
+
+
+
+Name
+Version
+Source
+Sink
+
+
+
+
+ Filesystem
+
+ Bounded and Unbounded Scan, Lookup
+ Streaming Sink, Batch Sink
+
+
+ Elasticsearch
+ 6.x & 7.x
+ Not supported
+ Streaming Sink, Batch Sink
+
+
+ Apache Kafka
+ 0.10+
+ Unbounded Scan
+ Streaming Sink, Batch Sink
+
+
+ JDBC
+
+ Bounded Scan, Lookup
+ Streaming Sink, Batch Sink
+
+
+ Apache
HBase
+ 1.4.x
+ Bounded Scan, Lookup
+ Streaming Sink, Batch Sink
+
+
+
+
+{% top %}
+
+How to use connectors
+
+
+Flink supports to use SQL CREATE TABLE statement to register a table. One can
define the name of the table, the schema of the table, the connector options
for connecting to an external system.
+
+The following code shows a full example of how to connect to Kafka for reading
Json records.
+
+
+
+{% highlight sql %}
+CREATE TABLE MyUserTable (
+ -- declare the schema of the table
+ `user` BIGINT,
+ message STRING,
+ ts TIMESTAMP,
+ proctime AS PROCTIME(), -- use computed column to define proctime attribute
+ WATERMARK FOR ts AS ts - INTERVAL '5' SECOND -- use WATERMARK statement to
define rowtime attribute
+) WITH (
+ -- declare the external system to connect to
+ 'connector' = 'kafka',
+ 'topic' = 'topic_name',
+ 'scan.startup.mode' = 'earliest-offset',
+ 'properties.bootstrap.servers' = 'localhost:9092',
+ 'format' = 'json' -- declare a format for this system
+)
+{% endhighlight %}
+
+
+
+In this ways the desired connection properties are converted into normalized,
string-based key-value pairs. So-called [table
factories](sourceSinks.html#define-a-tablefactory) create configured table
sources, table sinks, and corresponding formats from the key-value pairs. All
table factories that can be found via Java's [Service Provider Interfaces
(SPI)](https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html) are taken
into account when searching for exactly-one matching table factory.
+
+If no factory can be found or multiple factories match for the given
properties, an exception will be thrown with additional information about
considered factories and supported properties.
+
+{% top %}
+
+Schema Mapping
+
+
+The body clause of a SQL `CREATE TABLE` statement defines the names and types
of columns, constraints and watermarks. Flink doesn't hold the data, thus the
schema definition only declares how to map types from an external system to
Flink’s representation. The mapping may not be mapped by names, it depends on
the implementation of formats and connectors. For example, a MySQL database
table is mapped by field names (not case sensitive), and a CSV filesystem is
mapped by field order (field names can be arbitrary). This will be explained in
every connectors.
+
+The following example shows a simple schema without time attributes and
one-to-one field mapping of input/output to table columns.
+
+
+
+{% highlight sql %}
+CREATE TABLE MyTable (
+ MyField1 INT,
+ MyField2 STRING,
+ MyField3 BOOLEAN
+) WITH (
+ ...
+)
+{% endhighlight %}
+
+
+
+### Primary Key
+
+Primary key constraints tell that a column or a set of columns of a table are
unique and they do not contain nulls. Primary key uniquely identify a row in a
table.
+
+The primary key of
[jira] [Closed] (FLINK-17940) It will throw NullPointerException when write data with Avro format using new property key in SQL-Client
[
https://issues.apache.org/jira/browse/FLINK-17940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-17940.
---
Fix Version/s: (was: 1.12.0)
Resolution: Duplicate
> It will throw NullPointerException when write data with Avro format using new
> property key in SQL-Client
> -
>
> Key: FLINK-17940
> URL: https://issues.apache.org/jira/browse/FLINK-17940
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Table SQL / Client
>Affects Versions: 1.11.0
> Environment: Docker Environment:
> zookeeper:
> image: wurstmeister/zookeeper:3.4.6
> ports:
> - "2181:2181"
> kafka:
> image: wurstmeister/kafka:2.12-2.2.1
> ports:
> - "9092:9092"
> - "9094:9094"
> depends_on:
> zookeeper
> environment:
> - KAFKA_ADVERTISED_LISTENERS=INSIDE://:9094,OUTSIDE://localhost:9092
> - KAFKA_LISTENERS=INSIDE://:9094,OUTSIDE://:9092
> - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
> - KAFKA_INTER_BROKER_LISTENER_NAME=INSIDE
> - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
> - KAFKA_CREATE_TOPICS:"order_cnt:1:1,orders:1:1,currency:1:1"
> volumes:
> - /var/run/docker.sock:/var/run/docker.sock
>Reporter: Shengkai Fang
>Priority: Major
>
> For the following job:
> {noformat}
> create table csv(
> user_name VARCHAR, is_new BOOLEAN, content VARCHAR
> ) with (
> 'connector' = 'filesystem',
> 'path' = '/Users/ohmeatball/Work/flink-sql-etl/data-
> generator/src/main/resources/user.csv',
> 'format' = 'csv');
> -
> CREATE TABLE AvroTest (
> user_name VARCHAR, is_new BOOLEAN, content VARCHAR
> ) WITH (
> 'connector' = 'kafka', 'topic' = 'avro_from_csv',
> 'properties.zookeeper.connect' = 'localhost:2181',
> 'properties.bootstrap.servers' = 'localhost:9092', 'properties.group.id' =
> 'testGroup3', 'scan.startup.mode' = 'earliest-offset', 'format' = 'avro');
> -
> insert into AvroTest select user_name, is_new, content from csv;
> {noformat}
> The exception stack is following:
>
> {code:java}
> 2020-05-26 19:51:22,212 WARN org.apache.flink.runtime.taskmanager.Task
> [] - FileSystemTableSource(user_name, is_new, content) -> Sink:
> Sink(table=[default_catalog.default_database.AvroTest], fields=[user_name,
> is_new, content]) (1/1) (283a383f3ed93b051f56d4b5aca7dfb9) switched from
> RUNNING to FAILED.java.lang.RuntimeException: Failed to serialize row.at
> org.apache.flink.formats.avro.AvroRowDataSerializationSchema.serialize(AvroRowDataSerializationSchema.java:118)
> ~[flink-avro-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.formats.avro.AvroRowDataSerializationSchema.serialize(AvroRowDataSerializationSchema.java:63)
> ~[flink-avro-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.connectors.kafka.internals.KeyedSerializationSchemaWrapper.serializeValue(KeyedSerializationSchemaWrapper.java:51)
> ~[flink-sql-connector-kafka_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.invoke(FlinkKafkaProducer.java:775)
> ~[flink-sql-connector-kafka_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.invoke(FlinkKafkaProducer.java:98)
> ~[flink-sql-connector-kafka_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.invoke(TwoPhaseCommitSinkFunction.java:235)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:86)
> ~[flink-table-blink_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:672)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)
> ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]at
>
[jira] [Created] (FLINK-18078) E2E tests manually for Hive streaming dim join
Jingsong Lee created FLINK-18078: Summary: E2E tests manually for Hive streaming dim join Key: FLINK-18078 URL: https://issues.apache.org/jira/browse/FLINK-18078 Project: Flink Issue Type: Sub-task Components: Connectors / Hive Reporter: Jingsong Lee Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #12444: [FLINK-18067]Change default value of yarnMinAllocationMB from zero to DEFAULT_RM_S…
flinkbot edited a comment on pull request #12444: URL: https://github.com/apache/flink/pull/12444#issuecomment-637596755 ## CI report: * 16ec8b694dc07564b09b60040d20b0da9c0e80c4 UNKNOWN * 59c44869c548ce9b1930d5b742a6f2c256c5a5b5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2576) * c23450c9e7735ca7bad0ff97c37480de6c420d0e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-18077) E2E tests manually for Hive streaming source
Jingsong Lee created FLINK-18077: Summary: E2E tests manually for Hive streaming source Key: FLINK-18077 URL: https://issues.apache.org/jira/browse/FLINK-18077 Project: Flink Issue Type: Sub-task Components: Connectors / Hive Reporter: Jingsong Lee Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #12451: [FLINK-17857][e2e] Fix Kubernetes and docker e2e tests on Mac OS
flinkbot commented on pull request #12451: URL: https://github.com/apache/flink/pull/12451#issuecomment-637922585 ## CI report: * 4c93b68fa64dd913f53d69dad4eab467e78e6507 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-18025) E2E tests manually for Hive streaming sink
[ https://issues.apache.org/jira/browse/FLINK-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated FLINK-18025: - Description: - hive streaming sink failover - hive streaming sink job re-run - hive streaming sink without partition - ... was: - Hive streaming source - Hive streaming sink - Hive dim join > E2E tests manually for Hive streaming sink > -- > > Key: FLINK-18025 > URL: https://issues.apache.org/jira/browse/FLINK-18025 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive >Affects Versions: 1.11.0 >Reporter: Danny Chen >Priority: Major > Fix For: 1.11.0 > > > - hive streaming sink failover > - hive streaming sink job re-run > - hive streaming sink without partition > - ... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #12452: [FLINK-18056][fs-connector] Removing the remaining in-progress file on initialization for hadoop path-based writer
flinkbot commented on pull request #12452: URL: https://github.com/apache/flink/pull/12452#issuecomment-637922636 ## CI report: * 7cb10bd1269179bbe66f5b840c0f0d2494c66a55 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-18076) Sql client uses wrong class loader when parsing queries
[
https://issues.apache.org/jira/browse/FLINK-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124547#comment-17124547
]
Jark Wu commented on FLINK-18076:
-
>From the recently bug issues, it seems that it's urgent to add end-to-end
>tests for the new connectors and new DDLs.
> Sql client uses wrong class loader when parsing queries
> ---
>
> Key: FLINK-18076
> URL: https://issues.apache.org/jira/browse/FLINK-18076
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.11.0, 1.12.0
>Reporter: Dawid Wysakowicz
>Priority: Blocker
> Fix For: 1.11.0
>
>
> Sql-client when parsing queries does not use the user class loader from
> {{ExecutionContext}}. This makes it impossible to query any sources if the
> dependencies are added with {{-j}} flag.
> In order to reproduce it try querying e.g. KafkaDynamicSource with
> {code}
> CREATE TABLE MyUserTable (
>f0 BIGINT
> ) WITH (
> 'connector' = 'kafka',
> 'topic' = 'topic_name', -- required: topic name from which the table is read
> -- required: specify the Kafka server connection string
> 'properties.bootstrap.servers' = 'localhost:9092',
> -- required for Kafka source, optional for Kafka sink, specify consumer
> group
> 'properties.group.id' = 'testGroup',
> -- optional: valid modes are "earliest-offset", "latest-offset",
> "group-offsets", "specific-offsets" or "timestamp"
> 'scan.startup.mode' = 'earliest-offset',
>'format' = 'avro'
> );
> SELECT * FROM MyUserTable;
> {code}
> It give exception:
> {code}
> Exception in thread "main" org.apache.flink.table.client.SqlClientException:
> Unexpected exception. This is a bug. Please consider filing an issue.
> at org.apache.flink.table.client.SqlClient.main(SqlClient.java:213)
> Caused by: org.apache.flink.table.client.gateway.SqlExecutionException:
> Invalidate SQL statement.
> at
> org.apache.flink.table.client.cli.SqlCommandParser.parseBySqlParser(SqlCommandParser.java:95)
> at
> org.apache.flink.table.client.cli.SqlCommandParser.parse(SqlCommandParser.java:79)
> at
> org.apache.flink.table.client.cli.CliClient.parseCommand(CliClient.java:256)
> at org.apache.flink.table.client.cli.CliClient.open(CliClient.java:212)
> at org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:142)
> at org.apache.flink.table.client.SqlClient.start(SqlClient.java:114)
> at org.apache.flink.table.client.SqlClient.main(SqlClient.java:201)
> Caused by: org.apache.flink.table.api.ValidationException: Unable to create a
> source for reading table 'default_catalog.default_database.MyUserTable'.
> Table options are:
> 'connector'='kafka'
> 'format'='avro'
> 'properties.bootstrap.servers'='localhost:9092'
> 'properties.group.id'='testGroup'
> 'scan.startup.mode'='earliest-offset'
> 'topic'='topic_name'
> at
> org.apache.flink.table.factories.FactoryUtil.createTableSource(FactoryUtil.java:125)
> at
> org.apache.flink.table.planner.plan.schema.CatalogSourceTable.buildTableScan(CatalogSourceTable.scala:135)
> at
> org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.scala:78)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3492)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2415)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2102)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2051)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:661)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:642)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3345)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:568)
> at
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:164)
> at
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:151)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.toQueryOperation(SqlToOperationConverter.java:773)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convertSqlQuery(SqlToOperationConverter.java:745)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:238)
> at
> org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66)
> at
>
[GitHub] [flink] flinkbot edited a comment on pull request #12144: [FLINK-17384][flink-dist] support read hbase conf dir from flink.conf and change HBaseConfiguration construction.
flinkbot edited a comment on pull request #12144: URL: https://github.com/apache/flink/pull/12144#issuecomment-628475376 ## CI report: * d36b959fb16a91c15babda10dd884cbbdec58420 UNKNOWN * 3df97a6fdb130416fa447295f4ac7eb174be9cc4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2557) * 84f420d9043272150872fe4a32c2eaad2400a750 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2592) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-18025) E2E tests manually for Hive streaming sink
[ https://issues.apache.org/jira/browse/FLINK-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated FLINK-18025: - Summary: E2E tests manually for Hive streaming sink (was: E2E tests manually for Hive integrate streaming) > E2E tests manually for Hive streaming sink > -- > > Key: FLINK-18025 > URL: https://issues.apache.org/jira/browse/FLINK-18025 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive >Affects Versions: 1.11.0 >Reporter: Danny Chen >Priority: Major > Fix For: 1.11.0 > > > - Hive streaming source > - Hive streaming sink > - Hive dim join -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-18056) Hive file sink throws exception when the target in-progress file exists.
[
https://issues.apache.org/jira/browse/FLINK-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124542#comment-17124542
]
Yun Gao commented on FLINK-18056:
-
Hi [~sjwiesman] I'm a little confusing on the scenario since when rollover the
part-file on checkpoint, it should always write to a new file. Without failover
or restart the new part-file should not exist before and thus the corresponding
in-progress file should not be created. A single sink with parallelism > 1
should also not cause the conflict since the part-file name contains the
subtask index. Does the job also have multiple sink writing to the same Hive
table ? Or is it possible that the Hive table has been written by some jobs
previously but may left some hidden files not cleared when cleaning the
previous results ?
> Hive file sink throws exception when the target in-progress file exists.
>
>
> Key: FLINK-18056
> URL: https://issues.apache.org/jira/browse/FLINK-18056
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem, Connectors / Hive
>Reporter: Yun Gao
>Assignee: Yun Gao
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> Currently after failover or restart, the Hive file sink will try to overwrite
> the data since the last checkpoint, however, currently neither the
> in-progress file is deleted nor hive uses the overwritten mode, thus an
> exception occurs after restarting:
> {code:java}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException):
> failed to create file
> /user/hive/warehouse/datalake/dt=2020-06-01/hr=22/.part-0-10.inprogress for
> DFSClient_NONMAPREDUCE_-1017064593_62 for client 100.96.206.42 because
> current leaseholder is trying to recreate file.
> {code}
> The full stack of the exception is
> {code:java}
> org.apache.flink.connectors.hive.FlinkHiveException:
> org.apache.flink.table.catalog.exceptions.CatalogException: Failed to create
> Hive RecordWriter
> at
> org.apache.flink.connectors.hive.write.HiveWriterFactory.createRecordWriter(HiveWriterFactory.java:159)
> at
> org.apache.flink.connectors.hive.write.HiveBulkWriterFactory.create(HiveBulkWriterFactory.java:47)
> at
> org.apache.flink.formats.hadoop.bulk.HadoopPathBasedPartFileWriter$HadoopPathBasedBucketWriter.openNewInProgressFile(HadoopPathBasedPartFileWriter.java:234)
> at
> org.apache.flink.formats.hadoop.bulk.HadoopPathBasedPartFileWriter$HadoopPathBasedBucketWriter.openNewInProgressFile(HadoopPathBasedPartFileWriter.java:207)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.rollPartFile(Bucket.java:209)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.write(Bucket.java:200)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.Buckets.onElement(Buckets.java:284)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSinkHelper.onElement(StreamingFileSinkHelper.java:104)
> at
> org.apache.flink.table.filesystem.stream.StreamingFileWriter.processElement(StreamingFileWriter.java:118)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:672)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)
> at StreamExecCalc$16.processElement(Unknown Source)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:672)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)
> at
> org.apache.flink.table.runtime.operators.wmassigners.WatermarkAssignerOperator.processElement(WatermarkAssignerOperator.java:123)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> at
>
[GitHub] [flink] Jiayi-Liao commented on pull request #12444: [FLINK-18067]Change default value of yarnMinAllocationMB from zero to DEFAULT_RM_S…
Jiayi-Liao commented on pull request #12444: URL: https://github.com/apache/flink/pull/12444#issuecomment-637920335 @wangyang0918 Thanks for your review. I've addressed your comments and also update the description. Please take another look when you can spare time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #12452: [FLINK-18056][fs-connector] Removing the remaining in-progress file on initialization for hadoop path-based writer
flinkbot commented on pull request #12452: URL: https://github.com/apache/flink/pull/12452#issuecomment-637919303 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 7cb10bd1269179bbe66f5b840c0f0d2494c66a55 (Wed Jun 03 02:43:50 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] Jiayi-Liao commented on a change in pull request #12444: [FLINK-18067]Change default value of yarnMinAllocationMB from zero to DEFAULT_RM_S…
Jiayi-Liao commented on a change in pull request #12444:
URL: https://github.com/apache/flink/pull/12444#discussion_r434277415
##
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##
@@ -508,7 +508,9 @@ public void killCluster(ApplicationId applicationId) throws
FlinkException {
throw new YarnDeploymentException("Could not retrieve
information about free cluster resources.", e);
}
- final int yarnMinAllocationMB =
yarnConfiguration.getInt(YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_MB,
0);
+ final int yarnMinAllocationMB = yarnConfiguration.getInt(
Review comment:
I put in `validateClusterResources` because I thought it's a part of the
validation, but I'm also okay with your way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-18056) Hive file sink throws exception when the target in-progress file exists.
[
https://issues.apache.org/jira/browse/FLINK-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-18056:
---
Labels: pull-request-available (was: )
> Hive file sink throws exception when the target in-progress file exists.
>
>
> Key: FLINK-18056
> URL: https://issues.apache.org/jira/browse/FLINK-18056
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem, Connectors / Hive
>Reporter: Yun Gao
>Assignee: Yun Gao
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> Currently after failover or restart, the Hive file sink will try to overwrite
> the data since the last checkpoint, however, currently neither the
> in-progress file is deleted nor hive uses the overwritten mode, thus an
> exception occurs after restarting:
> {code:java}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException):
> failed to create file
> /user/hive/warehouse/datalake/dt=2020-06-01/hr=22/.part-0-10.inprogress for
> DFSClient_NONMAPREDUCE_-1017064593_62 for client 100.96.206.42 because
> current leaseholder is trying to recreate file.
> {code}
> The full stack of the exception is
> {code:java}
> org.apache.flink.connectors.hive.FlinkHiveException:
> org.apache.flink.table.catalog.exceptions.CatalogException: Failed to create
> Hive RecordWriter
> at
> org.apache.flink.connectors.hive.write.HiveWriterFactory.createRecordWriter(HiveWriterFactory.java:159)
> at
> org.apache.flink.connectors.hive.write.HiveBulkWriterFactory.create(HiveBulkWriterFactory.java:47)
> at
> org.apache.flink.formats.hadoop.bulk.HadoopPathBasedPartFileWriter$HadoopPathBasedBucketWriter.openNewInProgressFile(HadoopPathBasedPartFileWriter.java:234)
> at
> org.apache.flink.formats.hadoop.bulk.HadoopPathBasedPartFileWriter$HadoopPathBasedBucketWriter.openNewInProgressFile(HadoopPathBasedPartFileWriter.java:207)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.rollPartFile(Bucket.java:209)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.write(Bucket.java:200)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.Buckets.onElement(Buckets.java:284)
> at
> org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSinkHelper.onElement(StreamingFileSinkHelper.java:104)
> at
> org.apache.flink.table.filesystem.stream.StreamingFileWriter.processElement(StreamingFileWriter.java:118)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:672)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)
> at StreamExecCalc$16.processElement(Unknown Source)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:672)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)
> at
> org.apache.flink.table.runtime.operators.wmassigners.WatermarkAssignerOperator.processElement(WatermarkAssignerOperator.java:123)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:672)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)
> at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)
> at StreamExecCalc$2.processElement(Unknown Source)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:717)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:692)
> at
>
[GitHub] [flink] dianfu commented on a change in pull request #12422: [FLINK-17901][python] Add module interface in PyFlink
dianfu commented on a change in pull request #12422: URL: https://github.com/apache/flink/pull/12422#discussion_r434276134 ## File path: flink-python/pyflink/table/tests/test_table_environment_api.py ## @@ -628,6 +628,15 @@ def test_table_environment_with_blink_planner(self): self.assert_equals(results, ['2,hi,hello\n', '3,hello,hello\n']) +def test_unload_and_load_module(self): Review comment: Move it to class TableEnvironmentTest? ## File path: flink-python/pyflink/table/module.py ## @@ -0,0 +1,77 @@ + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from pyflink.java_gateway import get_gateway +from pyflink.table.function_definition import FunctionDefinition + +__all__ = ['Module', 'HiveModule'] + + +class Module(object): +""" +Modules define a set of metadata, including functions, user defined types, operators, rules, +etc. +Metadata from modules are regarded as built-in or system metadata that users can take advantages +of. +""" + +def __init__(self, j_module): +self._j_module = j_module + +@staticmethod +def _get(j_module): +if j_module.getClass().getName() == 'org.apache.flink.table.module.hive.HiveModule': +return HiveModule(j_hive_module=j_module) +else: +return Module(j_module) + +def list_functions(self): Review comment: This interfaces `list_functions` and `get_function_definition` are designed to used by: - Internally by the ModuleManager - Used to create custom Module So I think there is no need to expose them to Python users and we should remove them(and also the Python FunctionDefinition class). ## File path: flink-python/pyflink/table/table_environment.py ## @@ -148,6 +147,29 @@ def get_catalog(self, catalog_name): else: return None +def load_module(self, module_name, module): +""" +Loads a :class:`~pyflink.table.Module` under a unique name. Modules will be kept +in the loaded order. +ValidationException is thrown when there is already a module with the same name. + +:param module_name: Name of the :class:`~pyflink.table.Module`. +:type module_name: str +:param module: The module instance. +:type module: pyflink.table.Module Review comment: Add the version information ``` .. versionadded:: 1.12.0 ``` ## File path: flink-python/pyflink/table/table_environment.py ## @@ -148,6 +147,29 @@ def get_catalog(self, catalog_name): else: return None +def load_module(self, module_name, module): +""" +Loads a :class:`~pyflink.table.Module` under a unique name. Modules will be kept +in the loaded order. +ValidationException is thrown when there is already a module with the same name. + +:param module_name: Name of the :class:`~pyflink.table.Module`. +:type module_name: str +:param module: The module instance. +:type module: pyflink.table.Module +""" +self._j_tenv.loadModule(module_name, module._j_module) + +def unload_module(self, module_name): +""" +Unloads a :class:`~pyflink.table.Module` with given name. +ValidationException is thrown when there is no module with the given name. + +:param module_name: Name of the :class:`~pyflink.table.Module`. +:type module_name: str +""" +self._j_tenv.unloadModule(module_name) Review comment: ditto ## File path: flink-python/pyflink/table/module.py ## @@ -0,0 +1,77 @@ + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +#
[jira] [Updated] (FLINK-18021) Manually tests for 1.11 SQL features
[ https://issues.apache.org/jira/browse/FLINK-18021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-18021: --- Summary: Manually tests for 1.11 SQL features (was: Complement tests for 1.11 SQL) > Manually tests for 1.11 SQL features > > > Key: FLINK-18021 > URL: https://issues.apache.org/jira/browse/FLINK-18021 > Project: Flink > Issue Type: Task > Components: Table SQL / API >Affects Versions: 1.11.0 >Reporter: Danny Chen >Priority: Major > Fix For: 1.11.0 > > > This is an umbrella issue to collect all kinds of tests (e2e and ITCases) > that need to cover for 1.11 release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-18021) Manually tests for 1.11 SQL features
[ https://issues.apache.org/jira/browse/FLINK-18021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-18021: --- Priority: Critical (was: Major) > Manually tests for 1.11 SQL features > > > Key: FLINK-18021 > URL: https://issues.apache.org/jira/browse/FLINK-18021 > Project: Flink > Issue Type: Task > Components: Table SQL / API >Affects Versions: 1.11.0 >Reporter: Danny Chen >Priority: Critical > Fix For: 1.11.0 > > > This is an umbrella issue to collect all kinds of tests (e2e and ITCases) > that need to cover for 1.11 release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-18021) Manually tests for 1.11 SQL features
[ https://issues.apache.org/jira/browse/FLINK-18021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-18021: --- Priority: Blocker (was: Critical) > Manually tests for 1.11 SQL features > > > Key: FLINK-18021 > URL: https://issues.apache.org/jira/browse/FLINK-18021 > Project: Flink > Issue Type: Task > Components: Table SQL / API >Affects Versions: 1.11.0 >Reporter: Danny Chen >Priority: Blocker > Fix For: 1.11.0 > > > This is an umbrella issue to collect all kinds of tests (e2e and ITCases) > that need to cover for 1.11 release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
