[GitHub] [flink] godfreyhe commented on pull request #18785: [FLINK-26167][table-planner] Explicitly set the partitioner for the sql operators whose shuffle and sort are removed

2022-02-16 Thread GitBox


godfreyhe commented on pull request #18785:
URL: https://github.com/apache/flink/pull/18785#issuecomment-1041345782


   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18726: [FLINK-25983] Add API for configuring maximal watermark drift

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18726:
URL: https://github.com/apache/flink/pull/18726#issuecomment-1036101042


   
   ## CI report:
   
   * 318d5650f14a51a9f1aee5fe5f97a14d5d54bef6 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31630)
 
   * 0ee9c6a291b7d2a9bf83707eee0f8d8f06f05afb Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31640)
 
   * 255530e344dce93e506b9cfc1c3d2c15894c51b0 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18680: [FLINK-25583] Support compacting small files for FileSink.

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18680:
URL: https://github.com/apache/flink/pull/18680#issuecomment-1033512635


   
   ## CI report:
   
   * 2ffd741d316346745bd8fe754bb02a73cbf0d0ba Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31614)
 
   * 48412828133309fc4f4cddecfedcdfcbbbee8ebf Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31643)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-26148) Change the format of adaptive batch scheduler config option to "jobmanager.adaptive-batch-scheduler.XXX"

2022-02-16 Thread Lijie Wang (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493156#comment-17493156
 ] 

Lijie Wang commented on FLINK-26148:


Yes, I will update it soon.

> Change the format of adaptive batch scheduler config option to 
> "jobmanager.adaptive-batch-scheduler.XXX"
> 
>
> Key: FLINK-26148
> URL: https://issues.apache.org/jira/browse/FLINK-26148
> Project: Flink
>  Issue Type: Sub-task
>  Components: Runtime / Coordination
>Reporter: Lijie Wang
>Assignee: Lijie Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.15.0
>
>
> Change the format of adaptive batch scheduler config option to 
> {{jobmanager.adaptive-batch-scheduler.XXX}} to align the format with the 
> existing scheduler option (For example, 
> {{{}jobmanager.adaptive-scheduler.min-parallelism-increase{}}}).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26148) Change the format of adaptive batch scheduler config option to "jobmanager.adaptive-batch-scheduler.XXX"

2022-02-16 Thread Zhu Zhu (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493154#comment-17493154
 ] 

Zhu Zhu commented on FLINK-26148:
-

[~wanglijie95] would you update FLIP-187 for the configuration name change?

> Change the format of adaptive batch scheduler config option to 
> "jobmanager.adaptive-batch-scheduler.XXX"
> 
>
> Key: FLINK-26148
> URL: https://issues.apache.org/jira/browse/FLINK-26148
> Project: Flink
>  Issue Type: Sub-task
>  Components: Runtime / Coordination
>Reporter: Lijie Wang
>Assignee: Lijie Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.15.0
>
>
> Change the format of adaptive batch scheduler config option to 
> {{jobmanager.adaptive-batch-scheduler.XXX}} to align the format with the 
> existing scheduler option (For example, 
> {{{}jobmanager.adaptive-scheduler.min-parallelism-increase{}}}).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26121) ZooKeeperLeaderRetrievalConnectionHandlingTest.testNewLeaderAfterReconnectTriggersListenerNotification failed on azure

2022-02-16 Thread Matthias Pohl (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493153#comment-17493153
 ] 

Matthias Pohl commented on FLINK-26121:
---

There seems to be a gap of 10s in processing time while waiting for the 
leadership information. It's hard to tell here whether that's a AzureCI worker 
issue where the instance was just not operating for some time or whether there 
was no output due to the busy waiting.

> ZooKeeperLeaderRetrievalConnectionHandlingTest.testNewLeaderAfterReconnectTriggersListenerNotification
>  failed on azure
> --
>
> Key: FLINK-26121
> URL: https://issues.apache.org/jira/browse/FLINK-26121
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Assignee: Matthias Pohl
>Priority: Major
>  Labels: test-stability
>
> {code:java}
> 2022-02-11T21:43:35.4936452Z Feb 11 21:43:35 java.lang.AssertionError: The 
> TestingFatalErrorHandler caught an exception.
> 2022-02-11T21:43:35.4940444Z Feb 11 21:43:35  at 
> org.apache.flink.runtime.util.TestingFatalErrorHandlerResource.after(TestingFatalErrorHandlerResource.java:81)
> 2022-02-11T21:43:35.4941937Z Feb 11 21:43:35  at 
> org.apache.flink.runtime.util.TestingFatalErrorHandlerResource.access$300(TestingFatalErrorHandlerResource.java:36)
> 2022-02-11T21:43:35.4943249Z Feb 11 21:43:35  at 
> org.apache.flink.runtime.util.TestingFatalErrorHandlerResource$1.evaluate(TestingFatalErrorHandlerResource.java:60)
> 2022-02-11T21:43:35.4944745Z Feb 11 21:43:35  at 
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2022-02-11T21:43:35.4945682Z Feb 11 21:43:35  at 
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> 2022-02-11T21:43:35.4946655Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> 2022-02-11T21:43:35.4947847Z Feb 11 21:43:35  at 
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> 2022-02-11T21:43:35.4948876Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> 2022-02-11T21:43:35.4949842Z Feb 11 21:43:35  at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> 2022-02-11T21:43:35.4951142Z Feb 11 21:43:35  at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> 2022-02-11T21:43:35.4952153Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> 2022-02-11T21:43:35.4953115Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> 2022-02-11T21:43:35.4954068Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> 2022-02-11T21:43:35.4955003Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> 2022-02-11T21:43:35.4955981Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> 2022-02-11T21:43:35.4956930Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> 2022-02-11T21:43:35.4958008Z Feb 11 21:43:35  at 
> org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> 2022-02-11T21:43:35.4958899Z Feb 11 21:43:35  at 
> org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> 2022-02-11T21:43:35.4959774Z Feb 11 21:43:35  at 
> org.junit.runner.JUnitCore.run(JUnitCore.java:115)
> 2022-02-11T21:43:35.4960911Z Feb 11 21:43:35  at 
> org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
> 2022-02-11T21:43:35.4962095Z Feb 11 21:43:35  at 
> org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
> 2022-02-11T21:43:35.4963136Z Feb 11 21:43:35  at 
> org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
> 2022-02-11T21:43:35.4964275Z Feb 11 21:43:35  at 
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
> 2022-02-11T21:43:35.4965527Z Feb 11 21:43:35  at 
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
> 2022-02-11T21:43:35.4966787Z Feb 11 21:43:35  at 
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
> 2022-02-11T21:43:35.4968228Z Feb 11 21:43:35  at 
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
> 2022-02-11T21:43:35.4969485Z Feb 11 21:43:35  at 
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
> 2022-02-11T21:43:35.4970753Z Feb 11 21:4

[GitHub] [flink] flinkbot edited a comment on pull request #18786: [hotfix][test] add support for field assignability

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18786:
URL: https://github.com/apache/flink/pull/18786#issuecomment-1040539589


   
   ## CI report:
   
   * 199d94fca6df52f120633c8202b40409e1d7e794 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31615)
 
   * 84d45868a291322310ba3d321b580b8513b35031 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31642)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18680: [FLINK-25583] Support compacting small files for FileSink.

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18680:
URL: https://github.com/apache/flink/pull/18680#issuecomment-1033512635


   
   ## CI report:
   
   *  Unknown: [CANCELED](TBD) 
   * 2ffd741d316346745bd8fe754bb02a73cbf0d0ba Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31614)
 
   * 48412828133309fc4f4cddecfedcdfcbbbee8ebf UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18786: [hotfix][test] add support for field assignability

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18786:
URL: https://github.com/apache/flink/pull/18786#issuecomment-1040539589


   
   ## CI report:
   
   * 6fa8c290984d253ef7408abb6d23d49370c0f4b8 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31580)
 
   * 199d94fca6df52f120633c8202b40409e1d7e794 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31615)
 
   * 84d45868a291322310ba3d321b580b8513b35031 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18799: [FLINK-26145][docs] fix a kubernetes image that does not exist in the…

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18799:
URL: https://github.com/apache/flink/pull/18799#issuecomment-1041334465


   
   ## CI report:
   
   * 73640cb14e6a68cacc27b281f8d5508a5ffa21ea Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31641)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18726: [FLINK-25983] Add API for configuring maximal watermark drift

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18726:
URL: https://github.com/apache/flink/pull/18726#issuecomment-1036101042


   
   ## CI report:
   
   * 318d5650f14a51a9f1aee5fe5f97a14d5d54bef6 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31630)
 
   * 0ee9c6a291b7d2a9bf83707eee0f8d8f06f05afb Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31640)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-25000) Scala 2.12.7 doesn't compile on Java 17

2022-02-16 Thread Chesnay Schepler (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler updated FLINK-25000:
-
Description: 
Fails with "fails with /packages cannot be represented as URI" during 
compilation.

2.12.15 was working fine.

  was:
Fails with "fails with /packages cannot be represented as URI" during 
compilation.

2.12.5 was working fine.


> Scala 2.12.7 doesn't compile on Java 17
> ---
>
> Key: FLINK-25000
> URL: https://issues.apache.org/jira/browse/FLINK-25000
> Project: Flink
>  Issue Type: Sub-task
>  Components: API / Scala
>Reporter: Chesnay Schepler
>Priority: Major
>
> Fails with "fails with /packages cannot be represented as URI" during 
> compilation.
> 2.12.15 was working fine.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] wanglijie95 commented on a change in pull request #18757: [FLINK-25226][doc] Add documentation about the AdaptiveBatchScheduler

2022-02-16 Thread GitBox


wanglijie95 commented on a change in pull request #18757:
URL: https://github.com/apache/flink/pull/18757#discussion_r807768428



##
File path: docs/content/docs/deployment/adaptive_batch_scheduler.md
##
@@ -0,0 +1,63 @@
+---
+title: Adaptive Batch Scheduler
+weight: 5
+type: docs
+
+---
+
+
+## Adaptive Batch Scheduler
+
+The Adaptive Batch Scheduler can automatically decide parallelisms of job 
vertices for batch jobs. If a job vertex is not set with a parallelism, the 
scheduler will decide parallelism for the job vertex according to the size of 
its consumed datasets. This can bring many benefits:
+- Batch job users can be relieved from parallelism tuning
+- Automatically tuned parallelisms can be vertex level and can better fit 
consumed datasets which have a varying volume size every day
+- Vertices from SQL batch jobs can be assigned with different parallelisms 
which are automatically tuned
+
+### Usage
+
+To automatically decide parallelisms for job vertices through Adaptive Batch 
Scheduler, you need to:
+- Configure to use Adaptive Batch Scheduler.
+- Set the parallelism of job vertices to `-1`.
+  
+ Configure to use Adaptive Batch Scheduler
+To use Adaptive Batch Scheduler, you need to set the 
[`jobmanager.scheduler`]({{< ref "docs/deployment/config" 
>}}#jobmanager-scheduler) to `AdpaptiveBatch`. In addition, there are several 
optional config options that might need adjustment when using Adaptive Batch 
Scheduler:
+- [`jobmanager.scheduler.adaptive-batch.min-parallelism`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-min-parallelism): The lower bound of 
allowed parallelism to set adaptively
+- [`jobmanager.scheduler.adaptive-batch.max-parallelism`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-max-parallelism): The upper bound of 
allowed parallelism to set adaptively
+- [`jobmanager.scheduler.adaptive-batch.data-volume-per-task`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-data-volume-per-task): The size of data 
volume to expect each task instance to process
+- [`jobmanager.scheduler.adaptive-batch.source-parallelism.default`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-source-parallelism-default): The 
default parallelism of source vertices
+
+ Set the parallelism of job vertices to `-1`
+Adaptive Batch Scheduler will only decide parallelism for job vertices whose 
parallelism is not specified by users (parallelism is `-1`). So if you want the 
parallelism of vertices can be decided automatically, you should configure as 
follows:
+- Set `paralleims.default` to `-1`
+- Set `table.exec.resource.default-parallelism` to -1 in SQL jobs.
+- Don't call `setParallelism()` for operators in datastream jobs.
+
+### Performance tuning
+
+1. It's recommended to use `Sort Shuffle` and set 
[`taskmanager.network.memory.buffers-per-channel`]({{< ref 
"docs/deployment/config" >}}#taskmanager-network-memory-buffers-per-channel) to 
`0`. This can decouple the network memory consumption from parallelism, so for 
large scale jobs, the possibility of "Insufficient number of network buffers" 
error can be decreased.
+2. It's not recommended to configure an excessive value for 
[`jobmanager.scheduler.adaptive-batch.max-parallelism`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-max-parallelism), otherwise it will 
affect the performance. Because this option can affect the number of 
subpartitions produced by upstream tasks, excessive number of subpartitions may 
degrade the performance of hash shuffle and the performance of network 
transmission due to small packets.
+   


   
+### Limitations
+
+- **ALL-EDGES-BLOCKING batch jobs only**: The first version of Adaptive Batch 
Scheduler only supports ALL-EDGES-BLOCKING batch jobs only.

Review comment:
   +1 for @zhuzhurk 's comment. Just tell user adaptive batch scheduler 
only support the case where `execution.batch-shuffle-mode` is 
`ALL-EXCHANGES-BLOCKING`, and link to the config pages.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot commented on pull request #18799: [FLINK-26145][docs] fix a kubernetes image that does not exist in the…

2022-02-16 Thread GitBox


flinkbot commented on pull request #18799:
URL: https://github.com/apache/flink/pull/18799#issuecomment-1041334465


   
   ## CI report:
   
   * 73640cb14e6a68cacc27b281f8d5508a5ffa21ea UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18798: [FLINK-26177][tests] Disable PulsarSourceITCase rescaling tests temporarily

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18798:
URL: https://github.com/apache/flink/pull/18798#issuecomment-1041265347


   
   ## CI report:
   
   * 7b3b53570a9ac3f4b7159e7b1bf24eb7d29b2649 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31627)
 
   * d87a36d6c2290af48b79a325beeeb917fe3501ba Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31638)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (FLINK-26185) E2E Elasticsearch Tests should use the new Sink interface

2022-02-16 Thread Alexander Preuss (Jira)
Alexander Preuss created FLINK-26185:


 Summary: E2E Elasticsearch Tests should use the new Sink interface
 Key: FLINK-26185
 URL: https://issues.apache.org/jira/browse/FLINK-26185
 Project: Flink
  Issue Type: Bug
  Components: Connectors / ElasticSearch, Tests
Affects Versions: 1.15.0
Reporter: Alexander Preuss


Currently the E2E tests for Elasticsearch (test_streaming_elasticsearch.sh) is 
testing the old Sink interface implementation. As we are now moving to the new 
Sink interface, we should update the tests to test the new implementation



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18726: [FLINK-25983] Add API for configuring maximal watermark drift

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18726:
URL: https://github.com/apache/flink/pull/18726#issuecomment-1036101042


   
   ## CI report:
   
   * 318d5650f14a51a9f1aee5fe5f97a14d5d54bef6 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31630)
 
   * 0ee9c6a291b7d2a9bf83707eee0f8d8f06f05afb UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18756: [FLINK-26075][table-api][table-planner] Persist node configuration

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18756:
URL: https://github.com/apache/flink/pull/18756#issuecomment-1039112190


   
   ## CI report:
   
   * c7c123b2541e931272beb18f619bf8d8a418e471 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31572)
 
   * c018f4c7da0b607e45293323b568ca55d414cea7 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31637)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot commented on pull request #18799: [FLINK-26145][docs] fix a kubernetes image that does not exist in the…

2022-02-16 Thread GitBox


flinkbot commented on pull request #18799:
URL: https://github.com/apache/flink/pull/18799#issuecomment-1041332368


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 73640cb14e6a68cacc27b281f8d5508a5ffa21ea (Wed Feb 16 
10:20:48 UTC 2022)
   
✅no warnings
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18798: [FLINK-26177][tests] Disable PulsarSourceITCase rescaling tests temporarily

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18798:
URL: https://github.com/apache/flink/pull/18798#issuecomment-1041265347


   
   ## CI report:
   
   * 7b3b53570a9ac3f4b7159e7b1bf24eb7d29b2649 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31627)
 
   * d87a36d6c2290af48b79a325beeeb917fe3501ba UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (FLINK-25532) Provide Flink SQL CLI as Docker image

2022-02-16 Thread Konstantin Knauf (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Knauf resolved FLINK-25532.
--
Resolution: Fixed

Fixed via Documentation.

> Provide Flink SQL CLI as Docker image
> -
>
> Key: FLINK-25532
> URL: https://issues.apache.org/jira/browse/FLINK-25532
> Project: Flink
>  Issue Type: New Feature
>  Components: Table SQL / Client
>Reporter: Martijn Visser
>Assignee: Konstantin Knauf
>Priority: Major
>  Labels: pull-request-available
>
> Flink is currently available via as Docker images. However, the Flink SQL CLI 
> isn't available as a Docker image. We should also provide this. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-25532) Provide Flink SQL CLI as Docker image

2022-02-16 Thread Konstantin Knauf (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Knauf updated FLINK-25532:
-
Fix Version/s: 1.15.0

> Provide Flink SQL CLI as Docker image
> -
>
> Key: FLINK-25532
> URL: https://issues.apache.org/jira/browse/FLINK-25532
> Project: Flink
>  Issue Type: New Feature
>  Components: Table SQL / Client
>Reporter: Martijn Visser
>Assignee: Konstantin Knauf
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.15.0
>
>
> Flink is currently available via as Docker images. However, the Flink SQL CLI 
> isn't available as a Docker image. We should also provide this. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Assigned] (FLINK-25532) Provide Flink SQL CLI as Docker image

2022-02-16 Thread Konstantin Knauf (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Knauf reassigned FLINK-25532:


Assignee: Konstantin Knauf

> Provide Flink SQL CLI as Docker image
> -
>
> Key: FLINK-25532
> URL: https://issues.apache.org/jira/browse/FLINK-25532
> Project: Flink
>  Issue Type: New Feature
>  Components: Table SQL / Client
>Reporter: Martijn Visser
>Assignee: Konstantin Knauf
>Priority: Major
>  Labels: pull-request-available
>
> Flink is currently available via as Docker images. However, the Flink SQL CLI 
> isn't available as a Docker image. We should also provide this. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18756: [FLINK-26075][table-api][table-planner] Persist node configuration

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18756:
URL: https://github.com/apache/flink/pull/18756#issuecomment-1039112190


   
   ## CI report:
   
   * c7c123b2541e931272beb18f619bf8d8a418e471 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31572)
 
   * c018f4c7da0b607e45293323b568ca55d414cea7 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-26145) k8s docs jobmanager-pod-template artifacts-fetcher:latest image is not exist, we can use busybox to replace it

2022-02-16 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-26145:
---
Labels: pull-request-available  (was: )

> k8s docs jobmanager-pod-template  artifacts-fetcher:latest image is not 
> exist, we can use busybox to replace it 
> 
>
> Key: FLINK-26145
> URL: https://issues.apache.org/jira/browse/FLINK-26145
> Project: Flink
>  Issue Type: Bug
>  Components: Documentation
>Affects Versions: 1.15.0
>Reporter: jackylau
>Assignee: jackylau
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.15.0
>
> Attachments: image-2022-02-15-16-55-43-593.png
>
>
> !image-2022-02-15-16-54-42-861.png!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] knaufk merged pull request #18725: [FLINK-25532] Improve Documentation of Flink Docker-Compose

2022-02-16 Thread GitBox


knaufk merged pull request #18725:
URL: https://github.com/apache/flink/pull/18725


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18726: [FLINK-25983] Add API for configuring maximal watermark drift

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18726:
URL: https://github.com/apache/flink/pull/18726#issuecomment-1036101042


   
   ## CI report:
   
   * 318d5650f14a51a9f1aee5fe5f97a14d5d54bef6 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31630)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] liuyongvs opened a new pull request #18799: [FLINK-26145][docs] fix a kubernetes image that does not exist in the…

2022-02-16 Thread GitBox


liuyongvs opened a new pull request #18799:
URL: https://github.com/apache/flink/pull/18799


   ## What is the purpose of the change
   
   * fix docs*
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18611: [FLINK-24385][table] Introduce TRY_CAST

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18611:
URL: https://github.com/apache/flink/pull/18611#issuecomment-1028155639


   
   ## CI report:
   
   * f369013d63f0d367eab9744112e56bd7a8987177 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31566)
 
   * 5296f1f1e20d9e966cb207bb9dda3c3f7dafea38 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31635)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (FLINK-25980) remove unnecessary condition in IntervalJoinOperator

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-25980.
-
Fix Version/s: 1.15.0
   Resolution: Fixed

> remove unnecessary condition in IntervalJoinOperator
> 
>
> Key: FLINK-25980
> URL: https://issues.apache.org/jira/browse/FLINK-25980
> Project: Flink
>  Issue Type: Improvement
>  Components: API / DataStream
>Affects Versions: 1.12.7, 1.13.5, 1.14.2, 1.14.3
>Reporter: hongshu
>Assignee: hongshu
>Priority: Major
>  Labels: pull-request-available, pull_request_available
> Fix For: 1.15.0
>
>
> Condition 'currentWatermark != Long.MIN_VALUE' covered by subsequent 
> condition 'timestamp < currentWatermark' 
> org.apache.flink.streaming.api.operators.co.IntervalJoinOperator#isLate
> {code:java}
> private boolean isLate(long timestamp) {
> long currentWatermark = internalTimerService.currentWatermark();
> return currentWatermark != Long.MIN_VALUE && timestamp < currentWatermark;
> } {code}
> if currentWatermark == Long.MIN_VALUE, timestamp < currentWatermark it's also 
> return false, so condition currentWatermark != Long.MIN_VALUE is unnecessary
> We can use the following code directly
> {code:java}
> private boolean isLate(long timestamp) {
> long currentWatermark = internalTimerService.currentWatermark();
> return timestamp < currentWatermark;
> } {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18611: [FLINK-24385][table] Introduce TRY_CAST

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18611:
URL: https://github.com/apache/flink/pull/18611#issuecomment-1028155639


   
   ## CI report:
   
   * f369013d63f0d367eab9744112e56bd7a8987177 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31566)
 
   * 5296f1f1e20d9e966cb207bb9dda3c3f7dafea38 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-25980) remove unnecessary condition in IntervalJoinOperator

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493139#comment-17493139
 ] 

Fabian Paul commented on FLINK-25980:
-

Merged in master: ebfc7c8d599d561a7d18a9ebea23a3c8297d2e0c

> remove unnecessary condition in IntervalJoinOperator
> 
>
> Key: FLINK-25980
> URL: https://issues.apache.org/jira/browse/FLINK-25980
> Project: Flink
>  Issue Type: Improvement
>  Components: API / DataStream
>Affects Versions: 1.12.7, 1.13.5, 1.14.2, 1.14.3
>Reporter: hongshu
>Assignee: hongshu
>Priority: Major
>  Labels: pull-request-available, pull_request_available
>
> Condition 'currentWatermark != Long.MIN_VALUE' covered by subsequent 
> condition 'timestamp < currentWatermark' 
> org.apache.flink.streaming.api.operators.co.IntervalJoinOperator#isLate
> {code:java}
> private boolean isLate(long timestamp) {
> long currentWatermark = internalTimerService.currentWatermark();
> return currentWatermark != Long.MIN_VALUE && timestamp < currentWatermark;
> } {code}
> if currentWatermark == Long.MIN_VALUE, timestamp < currentWatermark it's also 
> return false, so condition currentWatermark != Long.MIN_VALUE is unnecessary
> We can use the following code directly
> {code:java}
> private boolean isLate(long timestamp) {
> long currentWatermark = internalTimerService.currentWatermark();
> return timestamp < currentWatermark;
> } {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] fapaul merged pull request #18647: [FLINK-25980][datastream] remove unnecessary condition

2022-02-16 Thread GitBox


fapaul merged pull request #18647:
URL: https://github.com/apache/flink/pull/18647


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] slinkydeveloper commented on a change in pull request #18611: [FLINK-24385][table] Introduce TRY_CAST

2022-02-16 Thread GitBox


slinkydeveloper commented on a change in pull request #18611:
URL: https://github.com/apache/flink/pull/18611#discussion_r807755945



##
File path: 
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/CastRuleProvider.java
##
@@ -107,6 +108,16 @@ public static boolean exists(LogicalType inputType, 
LogicalType targetType) {
 return resolve(inputType, targetType) != null;
 }
 
+/**
+ * Resolves the rule and returns the result of {@link 
CastRule#canFail(LogicalType,
+ * LogicalType)}. Fails with {@link NullPointerException} if the rule 
cannot be resolved.
+ */
+public static boolean canFail(LogicalType inputType, LogicalType 
targetType) {
+return Preconditions.checkNotNull(
+resolve(inputType, targetType), "Cast rule cannot be 
resolved")
+.canFail(inputType, targetType);

Review comment:
   I pushed in the last commit a consistent change to fix this, in order to 
merge the "can fail" algorithm with the rule matching algorithm.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Closed] (FLINK-25946) table-planner-loader jar NOTICE should list Scala

2022-02-16 Thread Chesnay Schepler (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler closed FLINK-25946.

Resolution: Fixed

> table-planner-loader jar NOTICE should list Scala
> -
>
> Key: FLINK-25946
> URL: https://issues.apache.org/jira/browse/FLINK-25946
> Project: Flink
>  Issue Type: Technical Debt
>  Components: Build System, Table SQL / Planner
>Affects Versions: 1.15.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.15.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18798: [FLINK-26177][tests] Disable PulsarSourceITCase rescaling tests temporarily

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18798:
URL: https://github.com/apache/flink/pull/18798#issuecomment-1041265347


   
   ## CI report:
   
   * 7b3b53570a9ac3f4b7159e7b1bf24eb7d29b2649 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31627)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18726: [FLINK-25983] Add API for configuring maximal watermark drift

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18726:
URL: https://github.com/apache/flink/pull/18726#issuecomment-1036101042


   
   ## CI report:
   
   * 9b9e4dc96990436cb5684646ed801b84f2357848 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31619)
 
   * 318d5650f14a51a9f1aee5fe5f97a14d5d54bef6 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31630)
 
   * 0ee9c6a291b7d2a9bf83707eee0f8d8f06f05afb UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Assigned] (FLINK-25958) OOME Checkpoints & Savepoints were shown as COMPLETE in Flink UI

2022-02-16 Thread Piotr Nowojski (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Piotr Nowojski reassigned FLINK-25958:
--

Assignee: Anton Kalashnikov

> OOME Checkpoints & Savepoints were shown as COMPLETE in Flink UI
> 
>
> Key: FLINK-25958
> URL: https://issues.apache.org/jira/browse/FLINK-25958
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Checkpointing
>Affects Versions: 1.15.0, 1.12.7, 1.13.5, 1.14.3
> Environment: Ververica Platform 2.6.2
> Flink 1.13.5
>Reporter: Victor Xu
>Assignee: Anton Kalashnikov
>Priority: Major
> Attachments: JIRA-1.jpg
>
>
> Flink job was running but the checkpoints & savepoints were failing all the 
> time due to OOM Exception. However, the Flink UI showed COMPLETE for those 
> checkpoints & savepoints.
> For example (checkpoint 39 & 40):
> {noformat}
> 2022-01-27 02:41:39,969 INFO  
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Triggering 
> checkpoint 39 (type=CHECKPOINT) @ 1643251299952 for job 
> ab2217e5ce144087bbddf6bd6c3
> 668eb.
> 2022-01-27 02:43:19,678 WARN  org.apache.flink.runtime.jobmaster.JobMaster    
>              [] - Error while processing AcknowledgeCheckpoint message
> org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete 
> the pending checkpoint 39. Failure reason: Failure to finalize checkpoint.
>         at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1227)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-stream2]
>         at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:1072)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-stream2]
>         at 
> org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$acknowledgeCheckpoint$1(ExecutionGraphHandler.java:89)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-stream2]
>         at 
> org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$processCheckpointCoordinatorMessage$3(ExecutionGraphHandler.java:119)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-s
> tream2]
>         at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
>         at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>  [?:?]
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  [?:?]
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  [?:?]
>         at java.lang.Thread.run(Thread.java:829) [?:?]
> Caused by: java.lang.IllegalArgumentException: Self-suppression not permitted
>         at java.lang.Throwable.addSuppressed(Throwable.java:1054) ~[?:?]
>         at 
> org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:627)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-stream2]
>         at 
> com.ververica.platform.flink.ha.kubernetes.KubernetesHaCheckpointStore.serializeCheckpoint(KubernetesHaCheckpointStore.java:204)
>  ~[vvp-flink-ha-kubernetes-flink113-1.4-20211013.09
> 1138-2.jar:?]
>         at 
> com.ververica.platform.flink.ha.kubernetes.KubernetesHaCheckpointStore.addCheckpoint(KubernetesHaCheckpointStore.java:83)
>  ~[vvp-flink-ha-kubernetes-flink113-1.4-20211013.091138-2.
> jar:?]
>         at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1209)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-stream2]
>         ... 9 more
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 2022-01-27 03:41:39,970 INFO  
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Triggering 
> checkpoint 40 (type=CHECKPOINT) @ 1643254899952 for job 
> ab2217e5ce144087bbddf6bd6c3
> 668eb.
> 2022-01-27 03:43:22,326 WARN  org.apache.flink.runtime.jobmaster.JobMaster    
>              [] - Error while processing AcknowledgeCheckpoint message
> org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete 
> the pending checkpoint 40. Failure reason: Failure to finalize checkpoint.
>         at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1227)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-stream2]
>         at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:1072)
>  ~[flink-dist_2.12-1.13.5-stream2.jar:1.13.5-stream2]
>         at 
> org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$acknowledgeCheckpoint$1(ExecutionGraphHandler.java:89)
>  ~[flink-

[GitHub] [flink] flinkbot edited a comment on pull request #18798: [FLINK-26177][tests] Disable PulsarSourceITCase rescaling tests temporarily

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18798:
URL: https://github.com/apache/flink/pull/18798#issuecomment-1041265347


   
   ## CI report:
   
   * 7b3b53570a9ac3f4b7159e7b1bf24eb7d29b2649 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31627)
 
   * d87a36d6c2290af48b79a325beeeb917fe3501ba UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18789: [FLINK-26168][runtime] The judgment of chainable ignores StreamExchangeMode when partitioner is ForwardForConsecutiveHashPartitioner

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18789:
URL: https://github.com/apache/flink/pull/18789#issuecomment-1040944258


   
   ## CI report:
   
   * cdab9f7d140989714263bc2a4e1a0908bb444a12 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31586)
 
   * 72851364f75d1b6e40ea1688408c3b85dccd6ec7 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31633)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-25571) Update Elasticsearch Sink to use decomposed interfaces

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-25571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493133#comment-17493133
 ] 

Fabian Paul commented on FLINK-25571:
-

Merged in master: 2d517950a9ae4a58b25f59fb6048f4c1a15225ac

> Update Elasticsearch Sink to use decomposed interfaces
> --
>
> Key: FLINK-25571
> URL: https://issues.apache.org/jira/browse/FLINK-25571
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / ElasticSearch
>Affects Versions: 1.15.0
>Reporter: Fabian Paul
>Assignee: Alexander Fedulov
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Resolved] (FLINK-25571) Update Elasticsearch Sink to use decomposed interfaces

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-25571.
-
Fix Version/s: 1.15.0
   Resolution: Fixed

> Update Elasticsearch Sink to use decomposed interfaces
> --
>
> Key: FLINK-25571
> URL: https://issues.apache.org/jira/browse/FLINK-25571
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / ElasticSearch
>Affects Versions: 1.15.0
>Reporter: Fabian Paul
>Assignee: Alexander Fedulov
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.15.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18785: [FLINK-26167][table-planner] Explicitly set the partitioner for the sql operators whose shuffle and sort are removed

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18785:
URL: https://github.com/apache/flink/pull/18785#issuecomment-1040524147


   
   ## CI report:
   
   * 50789f74f16c2bd3e42a61754f426877a3376056 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31608)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-26183) Support kubernetes-operator metrics using the Flink metric system

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26183:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Support kubernetes-operator metrics using the Flink metric system
> -
>
> Key: FLINK-26183
> URL: https://issues.apache.org/jira/browse/FLINK-26183
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Assignee: Matyas Orhidi
>Priority: Major
>
> We should leverage the existing Metric and Reporter infrastructure of Flink 
> to expose metrics of the Flink operator.
> Users should be able to pass configuration to the operator that will control 
> the metric registry (like for regular Flink jobs) and should be able to 
> access the built in metric reporter plugins.
>  
> Initially we should expose standard JVM metrics, and later we can add 
> operator specific metric groups if needed.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26181) Support manual savepoint triggering in the operator

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26181:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Support manual savepoint triggering in the operator
> ---
>
> Key: FLINK-26181
> URL: https://issues.apache.org/jira/browse/FLINK-26181
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> We should allow users to trigger savepoints manually. There have been 
> different approaches to implement this in different operators:
>  - savepoint generation field (Google/Spotify operator)
>  - trigger savepoint via resource annotation [~wangyang0918] - native 
> operator poc
> I think the annotation approach is more elegant but we should discuss this on 
> the ML before proceeding with the implementation



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] wanglijie95 commented on a change in pull request #18757: [FLINK-25226][doc] Add documentation about the AdaptiveBatchScheduler

2022-02-16 Thread GitBox


wanglijie95 commented on a change in pull request #18757:
URL: https://github.com/apache/flink/pull/18757#discussion_r807748519



##
File path: docs/content/docs/deployment/adaptive_batch_scheduler.md
##
@@ -0,0 +1,63 @@
+---
+title: Adaptive Batch Scheduler
+weight: 5
+type: docs
+
+---
+
+
+## Adaptive Batch Scheduler
+
+The Adaptive Batch Scheduler can automatically decide parallelisms of job 
vertices for batch jobs. If a job vertex is not set with a parallelism, the 
scheduler will decide parallelism for the job vertex according to the size of 
its consumed datasets. This can bring many benefits:
+- Batch job users can be relieved from parallelism tuning
+- Automatically tuned parallelisms can be vertex level and can better fit 
consumed datasets which have a varying volume size every day
+- Vertices from SQL batch jobs can be assigned with different parallelisms 
which are automatically tuned
+
+### Usage
+
+To automatically decide parallelisms for job vertices through Adaptive Batch 
Scheduler, you need to:
+- Configure to use Adaptive Batch Scheduler.
+- Set the parallelism of job vertices to `-1`.
+  
+ Configure to use Adaptive Batch Scheduler
+To use Adaptive Batch Scheduler, you need to set the 
[`jobmanager.scheduler`]({{< ref "docs/deployment/config" 
>}}#jobmanager-scheduler) to `AdpaptiveBatch`. In addition, there are several 
optional config options that might need adjustment when using Adaptive Batch 
Scheduler:
+- [`jobmanager.scheduler.adaptive-batch.min-parallelism`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-min-parallelism): The lower bound of 
allowed parallelism to set adaptively
+- [`jobmanager.scheduler.adaptive-batch.max-parallelism`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-max-parallelism): The upper bound of 
allowed parallelism to set adaptively
+- [`jobmanager.scheduler.adaptive-batch.data-volume-per-task`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-data-volume-per-task): The size of data 
volume to expect each task instance to process
+- [`jobmanager.scheduler.adaptive-batch.source-parallelism.default`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-source-parallelism-default): The 
default parallelism of source vertices
+
+ Set the parallelism of job vertices to `-1`
+Adaptive Batch Scheduler will only decide parallelism for job vertices whose 
parallelism is not specified by users (parallelism is `-1`). So if you want the 
parallelism of vertices can be decided automatically, you should configure as 
follows:
+- Set `paralleims.default` to `-1`
+- Set `table.exec.resource.default-parallelism` to -1 in SQL jobs.
+- Don't call `setParallelism()` for operators in datastream jobs.
+
+### Performance tuning
+
+1. It's recommended to use `Sort Shuffle` and set 
[`taskmanager.network.memory.buffers-per-channel`]({{< ref 
"docs/deployment/config" >}}#taskmanager-network-memory-buffers-per-channel) to 
`0`. This can decouple the network memory consumption from parallelism, so for 
large scale jobs, the possibility of "Insufficient number of network buffers" 
error can be decreased.

Review comment:
   +1 for this. I will add it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] fapaul merged pull request #18666: [FLINK-25571] Update Elasticsearch Sink to use decomposed interfaces

2022-02-16 Thread GitBox


fapaul merged pull request #18666:
URL: https://github.com/apache/flink/pull/18666


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18604: [FLINK-25926][Connectors][JDBC] Update org.postgresql:postgresql to 42.3.2

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18604:
URL: https://github.com/apache/flink/pull/18604#issuecomment-1027886359


   
   ## CI report:
   
   * 948f554e347ff5cb332d3273372718012d5b14fb Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31549)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] wanglijie95 commented on a change in pull request #18757: [FLINK-25226][doc] Add documentation about the AdaptiveBatchScheduler

2022-02-16 Thread GitBox


wanglijie95 commented on a change in pull request #18757:
URL: https://github.com/apache/flink/pull/18757#discussion_r807748121



##
File path: docs/content/docs/deployment/adaptive_batch_scheduler.md
##
@@ -0,0 +1,63 @@
+---
+title: Adaptive Batch Scheduler
+weight: 5
+type: docs
+
+---
+
+
+## Adaptive Batch Scheduler
+
+The Adaptive Batch Scheduler can automatically decide parallelisms of job 
vertices for batch jobs. If a job vertex is not set with a parallelism, the 
scheduler will decide parallelism for the job vertex according to the size of 
its consumed datasets. This can bring many benefits:
+- Batch job users can be relieved from parallelism tuning
+- Automatically tuned parallelisms can be vertex level and can better fit 
consumed datasets which have a varying volume size every day
+- Vertices from SQL batch jobs can be assigned with different parallelisms 
which are automatically tuned
+
+### Usage
+
+To automatically decide parallelisms for job vertices through Adaptive Batch 
Scheduler, you need to:
+- Configure to use Adaptive Batch Scheduler.
+- Set the parallelism of job vertices to `-1`.
+  
+ Configure to use Adaptive Batch Scheduler
+To use Adaptive Batch Scheduler, you need to set the 
[`jobmanager.scheduler`]({{< ref "docs/deployment/config" 
>}}#jobmanager-scheduler) to `AdpaptiveBatch`. In addition, there are several 
optional config options that might need adjustment when using Adaptive Batch 
Scheduler:
+- [`jobmanager.scheduler.adaptive-batch.min-parallelism`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-min-parallelism): The lower bound of 
allowed parallelism to set adaptively
+- [`jobmanager.scheduler.adaptive-batch.max-parallelism`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-max-parallelism): The upper bound of 
allowed parallelism to set adaptively
+- [`jobmanager.scheduler.adaptive-batch.data-volume-per-task`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-data-volume-per-task): The size of data 
volume to expect each task instance to process
+- [`jobmanager.scheduler.adaptive-batch.source-parallelism.default`]({{< ref 
"docs/deployment/config" 
>}}#jobmanager-scheduler-adaptive-batch-source-parallelism-default): The 
default parallelism of source vertices
+
+ Set the parallelism of job vertices to `-1`

Review comment:
   If users explicitly configure the `parallelism.default` (with a value > 
0) in `flink-conf`, but we override this value with `-1`, I think this may give 
the users a feeling that the configuration does not take effect. Maybe we can 
check the value of `parallelism.default` and then print an `ERROR` or  
`WARNING` log if the value > 0 ?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-26141) Support last-state upgrade mode

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26141:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Support last-state upgrade mode
> ---
>
> Key: FLINK-26141
> URL: https://issues.apache.org/jira/browse/FLINK-26141
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> The operator currently only implements savepoint and stateless upgrade 
> strategies for Flink jobs.
> We should investigate if we can provide last-state upgrade strategy that 
> would use the latest available checkpoint the make job upgrades instead of 
> savepoints.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26142) Integrate flink-kubernetes-operator repo with CI/CD

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26142:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Integrate flink-kubernetes-operator repo with CI/CD
> ---
>
> Key: FLINK-26142
> URL: https://issues.apache.org/jira/browse/FLINK-26142
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Assignee: Yang Wang
>Priority: Major
>
> We should be able to run the unit/integration tests on the existing Flink CI 
> infra and publish docker images for new builds to dockerhub



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26157) Containers Should Not Run As Root

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26157:
---
Component/s: Kubernetes Operator

> Containers Should Not Run As Root
> -
>
> Key: FLINK-26157
> URL: https://issues.apache.org/jira/browse/FLINK-26157
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Matyas Orhidi
>Assignee: Matyas Orhidi
>Priority: Major
>
> Processes in a container should not run as root. Create a user in the 
> Dockerfile with a known UID:GID (e.g. flink:flink)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26179) Support periodic savepointing in the operator

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26179:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Support periodic savepointing in the operator
> -
>
> Key: FLINK-26179
> URL: https://issues.apache.org/jira/browse/FLINK-26179
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> Automatic triggering of savepoints is a commonly requested feature. The 
> configuration should be part of the job spec.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26163) Refactor FlinkUtils#getEffectiveConfig into smaller pieces

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26163:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Refactor FlinkUtils#getEffectiveConfig into smaller pieces
> --
>
> Key: FLINK-26163
> URL: https://issues.apache.org/jira/browse/FLINK-26163
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Assignee: Biao Geng
>Priority: Major
>
> There is currently one very large method dealing with Configuration 
> management. This logic probably deserves it's own utility class and a few 
> more modular methods.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26140) Add basic handling mechanism to deal with job upgrade errors

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26140:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Add basic handling mechanism to deal with job upgrade errors
> 
>
> Key: FLINK-26140
> URL: https://issues.apache.org/jira/browse/FLINK-26140
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> There are various different ways how a stateful job upgrade can fail.
> For example:
> - Failure/timeout during savepoint
> - Incompatible state
> - Corrupted / not-found checkpoint
> - Error after restart
> We should allow some strategies for the user to declare how to handle the 
> different error scenarios (such as roll back to earlier state) and what 
> should be treated as a fatal error.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26139) Improve JobStatus tracking and handle different job states

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26139:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Improve JobStatus tracking and handle different job states
> --
>
> Key: FLINK-26139
> URL: https://issues.apache.org/jira/browse/FLINK-26139
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> Currently we do not handle any job status changes such as cancellations, 
> errors or job completions.
> We should introduce some mechanism to react and deal with these changes and 
> expose them in the status as they can potentially affect upgrades.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26138) Create controller test

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26138:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Create controller test
> --
>
> Key: FLINK-26138
> URL: https://issues.apache.org/jira/browse/FLINK-26138
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> Add test to validate the general controller flow like observe -> reconcile, 
> deletions etc.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26137) Create webhook REST api test

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26137:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Create webhook REST api test
> 
>
> Key: FLINK-26137
> URL: https://issues.apache.org/jira/browse/FLINK-26137
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> Add test to validate the webhook rest endpoint and make sure it returns the 
> expected responses, status codes etc.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26135) Separate job and deployment errors in FlinkDeployment status

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26135:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Separate job and deployment errors in FlinkDeployment status
> 
>
> Key: FLINK-26135
> URL: https://issues.apache.org/jira/browse/FLINK-26135
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>Priority: Major
>
> At the moment the controller does not validate or tolerate any deployment 
> errors such as incorrect configurations etc. Those will lead to an exception 
> loop in the reconcile logic.
> There are cases where the job deployment cannot be executed due to incorrect 
> configuration or other causes. In these cases the job can still be running 
> correctly so the job status should be OK but we should signal a deployment 
> error to the user that requires action.
> There should be a shared validation logic between the controller and the 
> webhook that should be applied whenever a new FlinkDeployment update is 
> received by the controller. If an error is detected in the controller, set 
> the deployment status to error with a useful message and leave the current 
> job running.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26136) Implement shared validation logic for FlinkDeployment objects

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26136:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Implement shared validation logic for FlinkDeployment objects
> -
>
> Key: FLINK-26136
> URL: https://issues.apache.org/jira/browse/FLINK-26136
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> At the moment there is only a very basic “placeholder” validation logic 
> implemented in the webhook module: 
> org.apache.flink.kubernetes.operator.admission.FlinkDeploymentValidator
> We should aim to validate parts of the FlinkDeployment that can be done 
> upfront, things like most common Flink config options, parallelism, resources 
> etc.
> As described in https://issues.apache.org/jira/browse/FLINK-26135 this 
> validation should be part of the flink-kubernetes-operator module.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26078) Initial Kubernetes Operator Prototype contribution

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26078:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Initial Kubernetes Operator Prototype contribution
> --
>
> Key: FLINK-26078
> URL: https://issues.apache.org/jira/browse/FLINK-26078
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>Priority: Major
>  Labels: pull-request-available
>
> This issue tracks the initial code contribution for the Flink Kubernetes 
> operator and any questions or concerns that might arise



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] dawidwys commented on a change in pull request #18745: [FLINK-24607] Make OperatorCoordinator closing sequence more robust.

2022-02-16 Thread GitBox


dawidwys commented on a change in pull request #18745:
URL: https://github.com/apache/flink/pull/18745#discussion_r807684936



##
File path: 
flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/ComponentClosingUtils.java
##
@@ -95,8 +100,94 @@ private ComponentClosingUtils() {}
 return future;
 }
 
+/**
+ * A util method that tries to shut down an {@link ExecutorService} 
elegantly within the given
+ * timeout. If the executor has not been shut down before it hits timeout 
or the thread is
+ * interrupted when waiting for the termination, a forceful shutdown will 
be attempted on the
+ * executor.
+ *
+ * @param executor the {@link ExecutorService} to shut down.
+ * @param timeout the timeout duration.
+ * @return true if the given executor has been successfully closed, false 
otherwise.
+ */
+@SuppressWarnings("ResultOfMethodCallIgnored")
+public static boolean tryShutdownExecutorElegantly(ExecutorService 
executor, Duration timeout) {
+try {
+executor.shutdown();
+executor.awaitTermination(timeout.toMillis(), 
TimeUnit.MILLISECONDS);
+} catch (InterruptedException ie) {
+// Let it go.
+}
+if (!executor.isTerminated()) {
+shutdownExecutorForcefully(executor, Duration.ZERO, false);
+}
+return executor.isTerminated();
+}
+
+/**
+ * Shutdown the given executor forcefully within the given timeout. The 
method returns if it is
+ * interrupted.
+ *
+ * @param executor the executor to shut down.
+ * @param timeout the timeout duration.
+ * @return true if the given executor is terminated, false otherwise.
+ */
+public static boolean shutdownExecutorForcefully(ExecutorService executor, 
Duration timeout) {
+return shutdownExecutorForcefully(executor, timeout, true);
+}
+
+/**
+ * Shutdown the given executor forcefully within the given timeout.
+ *
+ * @param executor the executor to shut down.
+ * @param timeout the timeout duration.
+ * @param interruptable when set to true, the method cannot be 
interrupted. Each interruption to

Review comment:
   I'd expect it to behave the other way around. If something is 
interruptable, I'd expect it can be properly interrupted. The way II understand 
it, now if the flag is true (interruptable), you can not interrupt this method.
   
   EDIT: Is it just that the javadoc is wrong?

##
File path: 
flink-runtime/src/main/java/org/apache/flink/runtime/source/coordinator/SourceCoordinatorContext.java
##
@@ -82,7 +84,7 @@ Licensed to the Apache Software Foundation (ASF) under one
 
 private static final Logger LOG = 
LoggerFactory.getLogger(SourceCoordinatorContext.class);
 
-private final ExecutorService coordinatorExecutor;
+private final ScheduledExecutorService coordinatorExecutor;

Review comment:
   nit: Could we create the `coordinatorExecutor` in the ctor? It's a bit 
weird now that we close a `coordinatorExecutor` that we are given from the 
outside of the class. As far as I checked it's always a new service created in 
the `SourceCoordinatorProvider`.

##
File path: 
flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/ComponentClosingUtils.java
##
@@ -95,8 +100,94 @@ private ComponentClosingUtils() {}
 return future;
 }
 
+/**
+ * A util method that tries to shut down an {@link ExecutorService} 
elegantly within the given
+ * timeout. If the executor has not been shut down before it hits timeout 
or the thread is
+ * interrupted when waiting for the termination, a forceful shutdown will 
be attempted on the
+ * executor.
+ *
+ * @param executor the {@link ExecutorService} to shut down.
+ * @param timeout the timeout duration.
+ * @return true if the given executor has been successfully closed, false 
otherwise.
+ */
+@SuppressWarnings("ResultOfMethodCallIgnored")
+public static boolean tryShutdownExecutorElegantly(ExecutorService 
executor, Duration timeout) {
+try {
+executor.shutdown();
+executor.awaitTermination(timeout.toMillis(), 
TimeUnit.MILLISECONDS);
+} catch (InterruptedException ie) {
+// Let it go.
+}
+if (!executor.isTerminated()) {
+shutdownExecutorForcefully(executor, Duration.ZERO, false);
+}
+return executor.isTerminated();
+}
+
+/**
+ * Shutdown the given executor forcefully within the given timeout. The 
method returns if it is
+ * interrupted.
+ *
+ * @param executor the executor to shut down.
+ * @param timeout the timeout duration.
+ * @return true if the given executor is terminated, false otherwise.
+ */
+public static boolean shutdownExecutorForcefully(ExecutorService executor, 
Duration timeout) {
+

[jira] [Commented] (FLINK-26178) Use the same enum for expected and observed jobstate (JobState / JobStatus.state)

2022-02-16 Thread Gyula Fora (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493129#comment-17493129
 ] 

Gyula Fora commented on FLINK-26178:


Done, thanks, will update all the tickets here

> Use the same enum for expected and observed jobstate (JobState / 
> JobStatus.state)
> -
>
> Key: FLINK-26178
> URL: https://issues.apache.org/jira/browse/FLINK-26178
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> We should consolidate these two and maybe even use Flink's own job state enum 
> here if makes sense



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-25963) FLIP-212: Introduce Flink Kubernetes Operator

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-25963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-25963:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> FLIP-212: Introduce Flink Kubernetes Operator
> -
>
> Key: FLINK-25963
> URL: https://issues.apache.org/jira/browse/FLINK-25963
> Project: Flink
>  Issue Type: Improvement
>  Components: Kubernetes Operator
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Major
>
> [https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator]
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26178) Use the same enum for expected and observed jobstate (JobState / JobStatus.state)

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora updated FLINK-26178:
---
Component/s: Kubernetes Operator
 (was: Deployment / Kubernetes)

> Use the same enum for expected and observed jobstate (JobState / 
> JobStatus.state)
> -
>
> Key: FLINK-26178
> URL: https://issues.apache.org/jira/browse/FLINK-26178
> Project: Flink
>  Issue Type: Sub-task
>  Components: Kubernetes Operator
>Reporter: Gyula Fora
>Priority: Major
>
> We should consolidate these two and maybe even use Flink's own job state enum 
> here if makes sense



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18798: [FLINK-26177][tests] Disable PulsarSourceITCase rescaling tests temporarily

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18798:
URL: https://github.com/apache/flink/pull/18798#issuecomment-1041265347


   
   ## CI report:
   
   * 7b3b53570a9ac3f4b7159e7b1bf24eb7d29b2649 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31627)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18789: [FLINK-26168][runtime] The judgment of chainable ignores StreamExchangeMode when partitioner is ForwardForConsecutiveHashPartitioner

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18789:
URL: https://github.com/apache/flink/pull/18789#issuecomment-1040944258


   
   ## CI report:
   
   * cdab9f7d140989714263bc2a4e1a0908bb444a12 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31586)
 
   * 72851364f75d1b6e40ea1688408c3b85dccd6ec7 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18604: [FLINK-25926][Connectors][JDBC] Update org.postgresql:postgresql to 42.3.2

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18604:
URL: https://github.com/apache/flink/pull/18604#issuecomment-1027886359


   
   ## CI report:
   
   * 948f554e347ff5cb332d3273372718012d5b14fb Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31549)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] MartijnVisser commented on pull request #18604: [FLINK-25926][Connectors][JDBC] Update org.postgresql:postgresql to 42.3.2

2022-02-16 Thread GitBox


MartijnVisser commented on pull request #18604:
URL: https://github.com/apache/flink/pull/18604#issuecomment-1041308638


   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-26184) Migrate StoreSink to Sink V2

2022-02-16 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-26184:
---
Labels: pull-request-available  (was: )

> Migrate StoreSink to Sink V2
> 
>
> Key: FLINK-26184
> URL: https://issues.apache.org/jira/browse/FLINK-26184
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table Store
>Reporter: Jingsong Lee
>Assignee: Jingsong Lee
>Priority: Major
>  Labels: pull-request-available
> Fix For: table-store-0.1.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26178) Use the same enum for expected and observed jobstate (JobState / JobStatus.state)

2022-02-16 Thread Chesnay Schepler (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493126#comment-17493126
 ] 

Chesnay Schepler commented on FLINK-26178:
--

You should be able to add it here: 
https://issues.apache.org/jira/projects/FLINK?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page

You can also do that by just typing the desired component into the tickets 
component field.

> Use the same enum for expected and observed jobstate (JobState / 
> JobStatus.state)
> -
>
> Key: FLINK-26178
> URL: https://issues.apache.org/jira/browse/FLINK-26178
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / Kubernetes
>Reporter: Gyula Fora
>Priority: Major
>
> We should consolidate these two and maybe even use Flink's own job state enum 
> here if makes sense



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink-table-store] JingsongLi opened a new pull request #22: [FLINK-26184] Migrate StoreSink to Sink V2

2022-02-16 Thread GitBox


JingsongLi opened a new pull request #22:
URL: https://github.com/apache/flink-table-store/pull/22


   This PR creates a new Sink interface: `GlobalCommittingSink`.
   ```
   /**
* The {@code GlobalCommitter} is responsible for creating and committing an 
aggregated committable,
* which we call global committable (see {@link #combine}).
*
* The {@code GlobalCommitter} runs with parallelism equal to 1.
*
* @param  The type of information needed to commit data staged by 
the sink
* @param  The type of the aggregated committable
*/
   public interface GlobalCommitter extends AutoCloseable {
   
   /** Find out which global committables need to be retried when 
recovering from the failure. */
   List filterRecoveredCommittables(List 
globalCommittables)
   throws IOException;
   
   /** Compute an aggregated committable from a list of committables. */
   GlobalCommT combine(long checkpointId, List committables) throws 
IOException;
   
   /** Commits the given {@link GlobalCommT}. */
   void commit(List globalCommittables) throws IOException, 
InterruptedException;
   }
   ```
   Because StoreSink committing needs to rely on checkpointId and needs to call 
the `filterRecoveredCommittables` interface carefully, a global commit 
mechanism similar to SinkV1 is created.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18798: [FLINK-26177][tests] Disable PulsarSourceITCase rescaling tests temporarily

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18798:
URL: https://github.com/apache/flink/pull/18798#issuecomment-1041265347


   
   ## CI report:
   
   * 7b3b53570a9ac3f4b7159e7b1bf24eb7d29b2649 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31627)
 
   * d87a36d6c2290af48b79a325beeeb917fe3501ba UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18718: [FLINK-25782] [docs] Translate datastream filesystem.md page into Chi…

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18718:
URL: https://github.com/apache/flink/pull/18718#issuecomment-1035898573


   
   ## CI report:
   
   * 8dbb8256563d48a7bdb13489d6670ae6215eebe1 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31611)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (FLINK-24246) Bump PulsarClient to 2.9.1 with better transaction management

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-24246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-24246.
-
Fix Version/s: 1.15.0
   Resolution: Fixed

> Bump PulsarClient to 2.9.1 with better transaction management
> -
>
> Key: FLINK-24246
> URL: https://issues.apache.org/jira/browse/FLINK-24246
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>  Labels: pull-request-available, stale-assigned
> Fix For: 1.15.0
>
>
> Pulsar 2.9.1 has been released, the hack for getting TxnID from Pulsar 
> Transaction instance could be removed after bumping flink-connector-pulsar's 
> pulsar-client-all to 2.9.1.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-24246) Bump PulsarClient to 2.9.1 with better transaction management

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-24246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493125#comment-17493125
 ] 

Fabian Paul commented on FLINK-24246:
-

Merged in master: 1602e4b7d26cf52cea993c410769b7b15a672aff

> Bump PulsarClient to 2.9.1 with better transaction management
> -
>
> Key: FLINK-24246
> URL: https://issues.apache.org/jira/browse/FLINK-24246
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>  Labels: pull-request-available, stale-assigned
>
> Pulsar 2.9.1 has been released, the hack for getting TxnID from Pulsar 
> Transaction instance could be removed after bumping flink-connector-pulsar's 
> pulsar-client-all to 2.9.1.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26020) Unified Pulsar Connector config model

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493124#comment-17493124
 ] 

Fabian Paul commented on FLINK-26020:
-

Merged in master: 36de46dc9cf3144a02a4f99c973bfb6b0db60c7e

> Unified Pulsar Connector config model
> -
>
> Key: FLINK-26020
> URL: https://issues.apache.org/jira/browse/FLINK-26020
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>
> PulsarClient has built-in config model named ClientConfigurationData, 
> ConsumerConfigurationData and ProducerConfigurationData. We don't want to 
> expose all the config options. And some config options could conflict with 
> each other.
> We decide to design a new config model based on Flink's Configuration. Which 
> could provide type checks and better integration with Flink Table.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26020) Unified Pulsar Connector config model

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul updated FLINK-26020:

Affects Version/s: 1.15.0

> Unified Pulsar Connector config model
> -
>
> Key: FLINK-26020
> URL: https://issues.apache.org/jira/browse/FLINK-26020
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>
> PulsarClient has built-in config model named ClientConfigurationData, 
> ConsumerConfigurationData and ProducerConfigurationData. We don't want to 
> expose all the config options. And some config options could conflict with 
> each other.
> We decide to design a new config model based on Flink's Configuration. Which 
> could provide type checks and better integration with Flink Table.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Resolved] (FLINK-26020) Unified Pulsar Connector config model

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-26020.
-
Fix Version/s: 1.15.0
   Resolution: Fixed

> Unified Pulsar Connector config model
> -
>
> Key: FLINK-26020
> URL: https://issues.apache.org/jira/browse/FLINK-26020
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>
> PulsarClient has built-in config model named ClientConfigurationData, 
> ConsumerConfigurationData and ProducerConfigurationData. We don't want to 
> expose all the config options. And some config options could conflict with 
> each other.
> We decide to design a new config model based on Flink's Configuration. Which 
> could provide type checks and better integration with Flink Table.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Resolved] (FLINK-26021) Pulsar topic deduplicated in both sink and source connector

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-26021.
-
Fix Version/s: 1.15.0
   Resolution: Fixed

> Pulsar topic deduplicated in both sink and source connector
> ---
>
> Key: FLINK-26021
> URL: https://issues.apache.org/jira/browse/FLINK-26021
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>
> Both topics and partitions are regarded as topics in Pulsar. We have to make 
> the topic configuration more robust for deduplicating the partitions and 
> topics.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] wanglijie95 commented on a change in pull request #18757: [FLINK-25226][doc] Add documentation about the AdaptiveBatchScheduler

2022-02-16 Thread GitBox


wanglijie95 commented on a change in pull request #18757:
URL: https://github.com/apache/flink/pull/18757#discussion_r807733497



##
File path: docs/content/docs/deployment/adaptive_batch_scheduler.md
##
@@ -0,0 +1,63 @@
+---
+title: Adaptive Batch Scheduler
+weight: 5
+type: docs
+
+---
+
+
+## Adaptive Batch Scheduler
+
+The Adaptive Batch Scheduler can automatically decide parallelisms of job 
vertices for batch jobs. If a job vertex is not set with a parallelism, the 
scheduler will decide parallelism for the job vertex according to the size of 
its consumed datasets. This can bring many benefits:
+- Batch job users can be relieved from parallelism tuning
+- Automatically tuned parallelisms can be vertex level and can better fit 
consumed datasets which have a varying volume size every day
+- Vertices from SQL batch jobs can be assigned with different parallelisms 
which are automatically tuned

Review comment:
   > What's the target audience? Does regular Flink user supposed to know 
what the job vertex is? Overall this page feels bit too low level 🤔.
   
   Thanks for pointing that out. Maybe `stage` is more appropriate?
   
   
   > On the other hand I don't think that other pages withing this section are 
all much better in this regard 🤔
   
   I'll check the rest content.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #18798: [FLINK-26177][tests] Disable PulsarSourceITCase rescaling tests temporarily

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18798:
URL: https://github.com/apache/flink/pull/18798#issuecomment-1041265347


   
   ## CI report:
   
   * 7b3b53570a9ac3f4b7159e7b1bf24eb7d29b2649 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31627)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-26021) Pulsar topic deduplicated in both sink and source connector

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493123#comment-17493123
 ] 

Fabian Paul commented on FLINK-26021:
-

Merged in master: a195f729ba4d1ffe657d82888b26ab9db9121ab8

> Pulsar topic deduplicated in both sink and source connector
> ---
>
> Key: FLINK-26021
> URL: https://issues.apache.org/jira/browse/FLINK-26021
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>
> Both topics and partitions are regarded as topics in Pulsar. We have to make 
> the topic configuration more robust for deduplicating the partitions and 
> topics.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26023) Create a Pulsar sink config model for matching ProducerConfigurationData

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493121#comment-17493121
 ] 

Fabian Paul commented on FLINK-26023:
-

Merged in master: 9bc8b0f37bec419bcdc4b8cdee3abf5320df5399

> Create a Pulsar sink config model for matching ProducerConfigurationData
> 
>
> Key: FLINK-26023
> URL: https://issues.apache.org/jira/browse/FLINK-26023
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18791: [FLINK-26169][coordination] There is a typo in the annotation for the WatermarkAlignmentParams class

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18791:
URL: https://github.com/apache/flink/pull/18791#issuecomment-1041038528


   
   ## CI report:
   
   * 5abd99fbc044163e5ca8b4ddfb6c33e55df06b44 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31606)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (FLINK-26023) Create a Pulsar sink config model for matching ProducerConfigurationData

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-26023.
-
Resolution: Fixed

> Create a Pulsar sink config model for matching ProducerConfigurationData
> 
>
> Key: FLINK-26023
> URL: https://issues.apache.org/jira/browse/FLINK-26023
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26178) Use the same enum for expected and observed jobstate (JobState / JobStatus.state)

2022-02-16 Thread Gyula Fora (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493122#comment-17493122
 ] 

Gyula Fora commented on FLINK-26178:


[~chesnay] thank you for jumping in, I was going to propose this. Do you know 
where to do this?

> Use the same enum for expected and observed jobstate (JobState / 
> JobStatus.state)
> -
>
> Key: FLINK-26178
> URL: https://issues.apache.org/jira/browse/FLINK-26178
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / Kubernetes
>Reporter: Gyula Fora
>Priority: Major
>
> We should consolidate these two and maybe even use Flink's own job state enum 
> here if makes sense



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26023) Create a Pulsar sink config model for matching ProducerConfigurationData

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul updated FLINK-26023:

Affects Version/s: 1.15.0

> Create a Pulsar sink config model for matching ProducerConfigurationData
> 
>
> Key: FLINK-26023
> URL: https://issues.apache.org/jira/browse/FLINK-26023
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26021) Pulsar topic deduplicated in both sink and source connector

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul updated FLINK-26021:

Affects Version/s: 1.15.0

> Pulsar topic deduplicated in both sink and source connector
> ---
>
> Key: FLINK-26021
> URL: https://issues.apache.org/jira/browse/FLINK-26021
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>
> Both topics and partitions are regarded as topics in Pulsar. We have to make 
> the topic configuration more robust for deduplicating the partitions and 
> topics.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26023) Create a Pulsar sink config model for matching ProducerConfigurationData

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul updated FLINK-26023:

Fix Version/s: 1.15.0

> Create a Pulsar sink config model for matching ProducerConfigurationData
> 
>
> Key: FLINK-26023
> URL: https://issues.apache.org/jira/browse/FLINK-26023
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26024) Create a PulsarSerializationSchema for better records serialization

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul updated FLINK-26024:

Affects Version/s: 1.15.0

> Create a PulsarSerializationSchema for better records serialization
> ---
>
> Key: FLINK-26024
> URL: https://issues.apache.org/jira/browse/FLINK-26024
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26163) Refactor FlinkUtils#getEffectiveConfig into smaller pieces

2022-02-16 Thread Gyula Fora (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493120#comment-17493120
 ] 

Gyula Fora commented on FLINK-26163:


Awesome, thanks [~bgeng777] 

> Refactor FlinkUtils#getEffectiveConfig into smaller pieces
> --
>
> Key: FLINK-26163
> URL: https://issues.apache.org/jira/browse/FLINK-26163
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / Kubernetes
>Reporter: Gyula Fora
>Assignee: Biao Geng
>Priority: Major
>
> There is currently one very large method dealing with Configuration 
> management. This logic probably deserves it's own utility class and a few 
> more modular methods.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[GitHub] [flink] flinkbot edited a comment on pull request #18725: [FLINK-25532] Improve Documentation of Flink Docker-Compose

2022-02-16 Thread GitBox


flinkbot edited a comment on pull request #18725:
URL: https://github.com/apache/flink/pull/18725#issuecomment-1036065440


   
   ## CI report:
   
   * b97354da0a65cd807b532a083f53c1c90b0c8a08 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31240)
 
   * 481c5adfee681c53c4d54ba2b2bd5a8746a615ad Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=31631)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (FLINK-26024) Create a PulsarSerializationSchema for better records serialization

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-26024.
-
Fix Version/s: 1.15.0
   Resolution: Fixed

> Create a PulsarSerializationSchema for better records serialization
> ---
>
> Key: FLINK-26024
> URL: https://issues.apache.org/jira/browse/FLINK-26024
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Assigned] (FLINK-26184) Migrate StoreSink to Sink V2

2022-02-16 Thread Jingsong Lee (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingsong Lee reassigned FLINK-26184:


Assignee: Jingsong Lee

> Migrate StoreSink to Sink V2
> 
>
> Key: FLINK-26184
> URL: https://issues.apache.org/jira/browse/FLINK-26184
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table Store
>Reporter: Jingsong Lee
>Assignee: Jingsong Lee
>Priority: Major
> Fix For: table-store-0.1.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26024) Create a PulsarSerializationSchema for better records serialization

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493119#comment-17493119
 ] 

Fabian Paul commented on FLINK-26024:
-

Merged in master: 0e72bfede70a00146f466b3e7491fc0f83eb6c41

> Create a PulsarSerializationSchema for better records serialization
> ---
>
> Key: FLINK-26024
> URL: https://issues.apache.org/jira/browse/FLINK-26024
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (FLINK-26022) Implement at-least-once and exactly-once Pulsar Sink

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul updated FLINK-26022:

Affects Version/s: 1.15.0

> Implement at-least-once and exactly-once Pulsar Sink
> 
>
> Key: FLINK-26022
> URL: https://issues.apache.org/jira/browse/FLINK-26022
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>
> Support both three types of DeliveryGuarantee in Pulsar sink.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Assigned] (FLINK-26163) Refactor FlinkUtils#getEffectiveConfig into smaller pieces

2022-02-16 Thread Gyula Fora (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gyula Fora reassigned FLINK-26163:
--

Assignee: Biao Geng

> Refactor FlinkUtils#getEffectiveConfig into smaller pieces
> --
>
> Key: FLINK-26163
> URL: https://issues.apache.org/jira/browse/FLINK-26163
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / Kubernetes
>Reporter: Gyula Fora
>Assignee: Biao Geng
>Priority: Major
>
> There is currently one very large method dealing with Configuration 
> management. This logic probably deserves it's own utility class and a few 
> more modular methods.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Resolved] (FLINK-26022) Implement at-least-once and exactly-once Pulsar Sink

2022-02-16 Thread Fabian Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-26022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabian Paul resolved FLINK-26022.
-
Fix Version/s: 1.15.0
   Resolution: Fixed

> Implement at-least-once and exactly-once Pulsar Sink
> 
>
> Key: FLINK-26022
> URL: https://issues.apache.org/jira/browse/FLINK-26022
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
> Fix For: 1.15.0
>
>
> Support both three types of DeliveryGuarantee in Pulsar sink.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (FLINK-26022) Implement at-least-once and exactly-once Pulsar Sink

2022-02-16 Thread Fabian Paul (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-26022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493118#comment-17493118
 ] 

Fabian Paul commented on FLINK-26022:
-

Merged in master: 136add5d0c9c5b9b2869a9ee194f78449065b18e

> Implement at-least-once and exactly-once Pulsar Sink
> 
>
> Key: FLINK-26022
> URL: https://issues.apache.org/jira/browse/FLINK-26022
> Project: Flink
>  Issue Type: Sub-task
>  Components: Connectors / Pulsar
>Affects Versions: 1.15.0
>Reporter: Yufan Sheng
>Assignee: Yufan Sheng
>Priority: Major
>
> Support both three types of DeliveryGuarantee in Pulsar sink.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


<    2   3   4   5   6   7   8   9   >