[flink] branch master updated (2231e1b -> 5984e7e)
This is an automated email from the ASF dual-hosted git repository.
zhuzh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 2231e1b [FLINK-15590][doc] add section for current catalog and
current database
add 5984e7e [FLINK-15307][runtime] Rename Subclasses of FailoverStrategy
No new revisions were added by this update.
Summary of changes:
.../AdaptedRestartPipelinedRegionStrategyNG.java | 12 +-
.../flip1/FailoverStrategyFactoryLoader.java | 8 +++
...rategy.java => RestartAllFailoverStrategy.java} | 8 +++
...=> RestartPipelinedRegionFailoverStrategy.java} | 12 +-
.../flip1/FailoverStrategyFactoryLoaderTest.java | 6 ++---
...st.java => RestartAllFailoverStrategyTest.java} | 6 ++---
...pelinedRegionFailoverStrategyBuildingTest.java} | 26 +++---
...estartPipelinedRegionFailoverStrategyTest.java} | 12 +-
.../runtime/scheduler/DefaultSchedulerTest.java| 4 ++--
.../ContinuousFileProcessingCheckpointITCase.java | 2 +-
.../StreamFaultToleranceTestBase.java | 10 -
11 files changed, 53 insertions(+), 53 deletions(-)
rename
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/{RestartAllStrategy.java
=> RestartAllFailoverStrategy.java} (88%)
rename
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/{RestartPipelinedRegionStrategy.java
=> RestartPipelinedRegionFailoverStrategy.java} (96%)
rename
flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/failover/flip1/{RestartAllStrategyTest.java
=> RestartAllFailoverStrategyTest.java} (90%)
rename
flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/failover/flip1/{RestartPipelinedRegionStrategyBuildingTest.java
=> RestartPipelinedRegionFailoverStrategyBuildingTest.java} (93%)
rename
flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/failover/flip1/{RestartPipelinedRegionStrategyTest.java
=> RestartPipelinedRegionFailoverStrategyTest.java} (96%)
[flink] branch release-1.10 updated: [FLINK-15590][doc] add section for current catalog and current database
This is an automated email from the ASF dual-hosted git repository.
bli pushed a commit to branch release-1.10
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.10 by this push:
new 4e39323 [FLINK-15590][doc] add section for current catalog and
current database
4e39323 is described below
commit 4e39323b9c52fcd30019f8be31ea5f37d7595e1d
Author: bowen.li
AuthorDate: Tue Jan 14 17:30:01 2020 -0800
[FLINK-15590][doc] add section for current catalog and current database
---
docs/dev/table/common.md| 13 +
docs/dev/table/common.zh.md | 13 +
2 files changed, 18 insertions(+), 8 deletions(-)
diff --git a/docs/dev/table/common.md b/docs/dev/table/common.md
index 6c7a3ed..eabc86a 100644
--- a/docs/dev/table/common.md
+++ b/docs/dev/table/common.md
@@ -297,7 +297,7 @@ Create Tables in the Catalog
A `TableEnvironment` maintains a map of catalogs of tables which are created
with an identifier. Each
identifier consists of 3 parts: catalog name, database name and object name.
If a catalog or database is not
-specified, the current default value will be used (see examples in the [Table
identifier expanding](#table-identifier-expanding) section).
+specified, the current default value will be used (see examples in the [Table
identifier expanding]({{ site.baseurl
}}/dev/table/common.html#table-identifier-expanding) section).
Tables can be either virtual (`VIEWS`) or regular (`TABLES`). `VIEWS` can be
created from an
existing `Table` object, usually the result of a Table API or SQL query.
`TABLES` describe
@@ -433,9 +433,14 @@ tableEnvironment.sqlUpdate("CREATE [TEMPORARY] TABLE
MyTable (...) WITH (...)")
### Expanding Table identifiers
-Tables are always registered with a 3 part identifier consisting of catalog,
database, and
-table name. The first two parts are optional and if they are not provided the
set default values will
-be used. Identifiers follow SQL requirements which means that they can be
escaped with a backtick character (`` ` ``).
+Tables are always registered with a 3-part identifier consisting of catalog,
database, and table name.
+
+Users can set one catalog and one database inside it to be the “current
catalog” and “current database”.
+With them, the first two parts in the 3-parts identifier mentioned above can
be optional - if they are not provided,
+the current catalog and current database will be referred. Users can switch
the current catalog and current database via
+table API or SQL.
+
+Identifiers follow SQL requirements which means that they can be escaped with
a backtick character (`` ` ``).
Additionally all SQL reserved keywords must be escaped.
diff --git a/docs/dev/table/common.zh.md b/docs/dev/table/common.zh.md
index 6c7a3ed..eabc86a 100644
--- a/docs/dev/table/common.zh.md
+++ b/docs/dev/table/common.zh.md
@@ -297,7 +297,7 @@ Create Tables in the Catalog
A `TableEnvironment` maintains a map of catalogs of tables which are created
with an identifier. Each
identifier consists of 3 parts: catalog name, database name and object name.
If a catalog or database is not
-specified, the current default value will be used (see examples in the [Table
identifier expanding](#table-identifier-expanding) section).
+specified, the current default value will be used (see examples in the [Table
identifier expanding]({{ site.baseurl
}}/dev/table/common.html#table-identifier-expanding) section).
Tables can be either virtual (`VIEWS`) or regular (`TABLES`). `VIEWS` can be
created from an
existing `Table` object, usually the result of a Table API or SQL query.
`TABLES` describe
@@ -433,9 +433,14 @@ tableEnvironment.sqlUpdate("CREATE [TEMPORARY] TABLE
MyTable (...) WITH (...)")
### Expanding Table identifiers
-Tables are always registered with a 3 part identifier consisting of catalog,
database, and
-table name. The first two parts are optional and if they are not provided the
set default values will
-be used. Identifiers follow SQL requirements which means that they can be
escaped with a backtick character (`` ` ``).
+Tables are always registered with a 3-part identifier consisting of catalog,
database, and table name.
+
+Users can set one catalog and one database inside it to be the “current
catalog” and “current database”.
+With them, the first two parts in the 3-parts identifier mentioned above can
be optional - if they are not provided,
+the current catalog and current database will be referred. Users can switch
the current catalog and current database via
+table API or SQL.
+
+Identifiers follow SQL requirements which means that they can be escaped with
a backtick character (`` ` ``).
Additionally all SQL reserved keywords must be escaped.
[flink] branch master updated (6bcb9bf -> 2231e1b)
This is an automated email from the ASF dual-hosted git repository. bli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 6bcb9bf [FLINK-15589][doc] remove beta tag from catalog and hive doc add 2231e1b [FLINK-15590][doc] add section for current catalog and current database No new revisions were added by this update. Summary of changes: docs/dev/table/common.md| 13 + docs/dev/table/common.zh.md | 13 + 2 files changed, 18 insertions(+), 8 deletions(-)
[flink] branch release-1.10 updated: [FLINK-15589][doc] remove beta tag from catalog and hive doc
This is an automated email from the ASF dual-hosted git repository. bli pushed a commit to branch release-1.10 in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/release-1.10 by this push: new af93ca2 [FLINK-15589][doc] remove beta tag from catalog and hive doc af93ca2 is described below commit af93ca2502e8f083fda99aea7b6e720a597d2a0a Author: bowen.li AuthorDate: Tue Jan 14 17:11:35 2020 -0800 [FLINK-15589][doc] remove beta tag from catalog and hive doc closes #10856 --- docs/dev/table/catalogs.md | 1 - docs/dev/table/catalogs.zh.md | 1 - docs/dev/table/hive/index.md| 1 - docs/dev/table/hive/index.zh.md | 1 - 4 files changed, 4 deletions(-) diff --git a/docs/dev/table/catalogs.md b/docs/dev/table/catalogs.md index 07f5958..79972e1 100644 --- a/docs/dev/table/catalogs.md +++ b/docs/dev/table/catalogs.md @@ -1,6 +1,5 @@ --- title: "Catalogs" -is_beta: true nav-parent_id: tableapi nav-pos: 80 --- diff --git a/docs/dev/table/catalogs.zh.md b/docs/dev/table/catalogs.zh.md index e0897f6..e07e840 100644 --- a/docs/dev/table/catalogs.zh.md +++ b/docs/dev/table/catalogs.zh.md @@ -1,6 +1,5 @@ --- title: "Catalogs" -is_beta: true nav-parent_id: tableapi nav-pos: 80 --- diff --git a/docs/dev/table/hive/index.md b/docs/dev/table/hive/index.md index c054843..f15b280 100644 --- a/docs/dev/table/hive/index.md +++ b/docs/dev/table/hive/index.md @@ -3,7 +3,6 @@ title: "Hive Integration" nav-id: hive_tableapi nav-parent_id: tableapi nav-pos: 100 -is_beta: true nav-show_overview: true ---
[flink] branch master updated (7513d9b -> 6bcb9bf)
This is an automated email from the ASF dual-hosted git repository. bli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 7513d9b [FLINK-15517][Runtime][Configuration][Network] Use back 'network' in 'shuffle' memory config option names add 6bcb9bf [FLINK-15589][doc] remove beta tag from catalog and hive doc No new revisions were added by this update. Summary of changes: docs/dev/table/catalogs.md | 1 - docs/dev/table/catalogs.zh.md | 1 - docs/dev/table/hive/index.md| 1 - docs/dev/table/hive/index.zh.md | 1 - 4 files changed, 4 deletions(-)
[flink-web] 01/02: Add part 1 of application patterns blog post series.
This is an automated email from the ASF dual-hosted git repository. fhueske pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git commit 455f6269c26a57c2a6502ae603a063e502a382e5 Author: Alexander Fedulov <1492164+afedu...@users.noreply.github.com> AuthorDate: Mon Jan 6 12:32:19 2020 +0100 Add part 1 of application patterns blog post series. This closes #289. --- _posts/2020-01-15-demo-fraud-detection.md | 222 + .../architecture.png | Bin 0 -> 214660 bytes .../2019-11-19-demo-fraud-detection/end-to-end.png | Bin 0 -> 195993 bytes .../shuffle_function_1.png | Bin 0 -> 152296 bytes img/blog/2019-11-19-demo-fraud-detection/ui.png| Bin 0 -> 1296038 bytes 5 files changed, 222 insertions(+) diff --git a/_posts/2020-01-15-demo-fraud-detection.md b/_posts/2020-01-15-demo-fraud-detection.md new file mode 100644 index 000..02511db --- /dev/null +++ b/_posts/2020-01-15-demo-fraud-detection.md @@ -0,0 +1,222 @@ +--- +layout: post +title: "Advanced Flink Application Patterns Vol.1: +Case Study of a Fraud Detection System" +date: 2020-01-15T12:00:00.000Z +authors: +- alex: + name: "Alexander Fedulov" + twitter: "alex_fedulov" +categories: news +excerpt: In this series of blog posts you will learn about three powerful Flink patterns for building streaming applications. +--- + +In this series of blog posts you will learn about three powerful Flink patterns for building streaming applications: + + - Dynamic updates of application logic + - Dynamic data partitioning (shuffle), controlled at runtime + - Low latency alerting based on custom windowing logic (without using the window API) + +These patterns expand the possibilities of what is achievable with statically defined data flows and provide the building blocks to fulfill complex business requirements. + +**Dynamic updates of application logic** allow Flink jobs to change at runtime, without downtime from stopping and resubmitting the code. + +**Dynamic data partitioning** provides the ability to change how events are distributed and grouped by Flink at runtime. Such functionality often becomes a natural requirement when building jobs with dynamically reconfigurable application logic. + +**Custom window management** demonstrates how you can utilize the low level [process function API](https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/process_function.html), when the native [window API](https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html) is not exactly matching your requirements. Specifically, you will learn how to implement low latency alerting on windows and how to limit state growth with timers. + +These patterns build on top of core Flink functionality, however, they might not be immediately apparent from the framework's documentation as explaining and presenting the motivation behind them is not always trivial without a concrete use case. That is why we will showcase these patterns with a practical example that offers a real-world usage scenario for Apache Flink — a _Fraud Detection_ engine. +We hope that this series will place these powerful approaches into your tool belt and enable you to take on new and exciting tasks. + +In the first blog post of the series we will look at the high-level architecture of the demo application, describe its components and their interactions. We will then deep dive into the implementation details of the first pattern in the series - **dynamic data partitioning**. + + +You will be able to run the full Fraud Detection Demo application locally and look into the details of the implementation by using the accompanying GitHub repository. + +### Fraud Detection Demo + +The full source code for our fraud detection demo is open source and available online. To run it locally, check out the following repository and follow the steps in the README: + +[https://github.com/afedulov/fraud-detection-demo](https://github.com/afedulov/fraud-detection-demo) + +You will see the demo is a self-contained application - it only requires `docker` and `docker-compose` to be built from sources and includes the following components: + + - Apache Kafka (message broker) with ZooKeeper + - Apache Flink ([application cluster](https://ci.apache.org/projects/flink/flink-docs-stable/concepts/glossary.html#flink-application-cluster)) + - Fraud Detection Web App + +The high-level goal of the Fraud Detection engine is to consume a stream of financial transactions and evaluate them against a set of rules. These rules are subject to frequent changes and tweaks. In a real production system, it is important to be able to add and remove them at runtime, without incurring an expensive penalty of stopping and restarting the job. + +When you navigate to the demo URL in your browser, you will be presented with the f
[flink-web] 02/02: Rebuild website
This is an automated email from the ASF dual-hosted git repository. fhueske pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git commit 1c63ea08d295e692406762812b82969137db7db3 Author: Fabian Hueske AuthorDate: Wed Jan 15 17:29:26 2020 +0100 Rebuild website --- content/blog/feed.xml | 212 ++ content/blog/index.html| 45 ++- content/blog/page10/index.html | 32 +- content/blog/page2/index.html | 45 ++- content/blog/page3/index.html | 45 ++- content/blog/page4/index.html | 47 ++- content/blog/page5/index.html | 47 ++- content/blog/page6/index.html | 47 ++- content/blog/page7/index.html | 45 ++- content/blog/page8/index.html | 43 +- content/blog/page9/index.html | 45 ++- .../architecture.png | Bin 0 -> 214660 bytes .../2019-11-19-demo-fraud-detection/end-to-end.png | Bin 0 -> 195993 bytes .../shuffle_function_1.png | Bin 0 -> 152296 bytes .../blog/2019-11-19-demo-fraud-detection/ui.png| Bin 0 -> 1296038 bytes content/index.html | 6 +- content/news/2020/01/15/demo-fraud-detection.html | 443 + content/zh/index.html | 6 +- 18 files changed, 963 insertions(+), 145 deletions(-) diff --git a/content/blog/feed.xml b/content/blog/feed.xml index c044975..653680c 100644 --- a/content/blog/feed.xml +++ b/content/blog/feed.xml @@ -7,6 +7,218 @@ https://flink.apache.org/blog/feed.xml"; rel="self" type="application/rss+xml" /> +Advanced Flink Application Patterns Vol.1: Case Study of a Fraud Detection System +In this series of blog posts you will learn about three powerful Flink patterns for building streaming applications:
+ ++
+ +- Dynamic updates of application logic
+- Dynamic data partitioning (shuffle), controlled at runtime
+- Low latency alerting based on custom windowing logic (without using the window API)
+These patterns expand the possibilities of what is achievable with statically defined data flows and provide the building blocks to fulfill complex business requirements.
+ +Dynamic updates of application logic allow Flink jobs to change at runtime, without downtime from stopping and resubmitting the code.
+
+Dynamic data partitioning provides the ability to change how events are distributed and grouped by Flink at runtime. Such functionality often becomes a natural requirement when building jobs with dynamically reconfigurable application logic.
+
+Custom window management demonstrates how you can utilize the low level process function API, when the native window API is not exactly matching your requirements. Specifically, you will learn how to impl [...] + +These patterns build on top of core Flink functionality, however, they might not be immediately apparent from the framework’s documentation as explaining and presenting the motivation behind them is not always trivial without a concrete use case. That is why we will showcase these patterns with a practical example that offers a real-world usage scenario for Apache Flink — a Fraud Detection engine. +We hope that this series will place these powerful approaches into your tool belt and enable you to take on new and exciting tasks.
+ +In the first blog post of the series we will look at the high-level architecture of the demo application, describe its components and their interactions. We will then deep dive into the implementation details of the first pattern in the series - dynamic data partitioning.
+ +You will be able to run the full Fraud Detection Demo application locally and look into the details of the implementation by using the accompanying GitHub repository.
+ +Fraud Detection Demo
+ +The full source code for our fraud detection demo is open source and available online. To run it locally, check out the following repository and follow the steps in the README:
+ +https://github.com/afedulov/fraud-detection-demo
; + +You will see the demo is a self-contained application - it only requires
+ +dockeranddocker-composeto be built from sources and includes the following components:+
- Apache Kafka (message bro
[flink-web] branch asf-site updated (f78b71e -> 1c63ea0)
This is an automated email from the ASF dual-hosted git repository. fhueske pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git. from f78b71e Rebuild website new 455f626 Add part 1 of application patterns blog post series. new 1c63ea0 Rebuild website The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _posts/2020-01-15-demo-fraud-detection.md | 222 +++ content/blog/feed.xml | 212 ++ content/blog/index.html| 45 ++- content/blog/page10/index.html | 32 +- content/blog/page2/index.html | 45 ++- content/blog/page3/index.html | 45 ++- content/blog/page4/index.html | 47 ++- content/blog/page5/index.html | 47 ++- content/blog/page6/index.html | 47 ++- content/blog/page7/index.html | 45 ++- content/blog/page8/index.html | 43 +- content/blog/page9/index.html | 45 ++- .../architecture.png | Bin 0 -> 214660 bytes .../2019-11-19-demo-fraud-detection/end-to-end.png | Bin 0 -> 195993 bytes .../shuffle_function_1.png | Bin 0 -> 152296 bytes .../blog/2019-11-19-demo-fraud-detection/ui.png| Bin 0 -> 1296038 bytes content/index.html | 6 +- content/news/2020/01/15/demo-fraud-detection.html | 443 + content/zh/index.html | 6 +- .../architecture.png | Bin 0 -> 214660 bytes .../2019-11-19-demo-fraud-detection/end-to-end.png | Bin 0 -> 195993 bytes .../shuffle_function_1.png | Bin 0 -> 152296 bytes img/blog/2019-11-19-demo-fraud-detection/ui.png| Bin 0 -> 1296038 bytes 23 files changed, 1185 insertions(+), 145 deletions(-) create mode 100644 _posts/2020-01-15-demo-fraud-detection.md create mode 100644 content/img/blog/2019-11-19-demo-fraud-detection/architecture.png create mode 100644 content/img/blog/2019-11-19-demo-fraud-detection/end-to-end.png create mode 100644 content/img/blog/2019-11-19-demo-fraud-detection/shuffle_function_1.png create mode 100644 content/img/blog/2019-11-19-demo-fraud-detection/ui.png create mode 100644 content/news/2020/01/15/demo-fraud-detection.html create mode 100644 img/blog/2019-11-19-demo-fraud-detection/architecture.png create mode 100644 img/blog/2019-11-19-demo-fraud-detection/end-to-end.png create mode 100644 img/blog/2019-11-19-demo-fraud-detection/shuffle_function_1.png create mode 100644 img/blog/2019-11-19-demo-fraud-detection/ui.png
[flink] branch release-1.10 updated (8acd60b -> a6023e6)
This is an automated email from the ASF dual-hosted git repository. azagrebin pushed a change to branch release-1.10 in repository https://gitbox.apache.org/repos/asf/flink.git. from 8acd60b [FLINK-15327][runtime] No warning of InterruptedException during cancel. add a6023e6 [FLINK-15517][Runtime][Configuration][Network] Use back 'network' in 'shuffle' memory config option names No new revisions were added by this update. Summary of changes: .../task_manager_memory_configuration.html | 44 +++--- docs/ops/config.md | 6 +- docs/ops/config.zh.md | 6 +- docs/ops/deployment/yarn_setup.md | 2 +- docs/ops/deployment/yarn_setup.zh.md | 2 +- .../org/apache/flink/client/cli/ExecutorCLI.java | 2 +- .../flink/configuration/ConfigurationUtils.java| 4 +- .../NettyShuffleEnvironmentOptions.java| 10 +- .../flink/configuration/TaskManagerOptions.java| 40 +++--- flink-dist/src/main/resources/flink-conf.yaml | 6 +- .../test-scripts/test_batch_allround.sh| 4 +- .../test-scripts/test_heavy_deployment.sh | 4 +- .../test_high_parallelism_iterations.sh| 4 +- .../flink/kubernetes/KubernetesUtilsTest.java | 2 +- .../runtime/clusterframework/BootstrapTools.java | 2 +- .../clusterframework/TaskExecutorResourceSpec.java | 20 +-- .../TaskExecutorResourceUtils.java | 160 ++--- .../clusterframework/types/ResourceProfile.java| 50 +++ .../io/network/NettyShuffleServiceFactory.java | 2 +- .../io/network/buffer/NetworkBufferPool.java | 6 +- .../minicluster/MiniClusterConfiguration.java | 8 +- .../runtime/shuffle/ShuffleEnvironmentContext.java | 10 +- .../runtime/taskexecutor/TaskManagerServices.java | 2 +- .../TaskManagerServicesConfiguration.java | 4 +- .../NettyShuffleEnvironmentConfiguration.java | 14 +- .../clusterframework/BootstrapToolsTest.java | 2 +- .../TaskExecutorResourceUtilsTest.java | 142 +- .../types/ResourceProfileTest.java | 42 +++--- .../jobmaster/slotpool/SlotSharingManagerTest.java | 6 +- .../minicluster/MiniClusterConfigurationTest.java | 4 +- .../taskexecutor/TaskManagerRunnerStartupTest.java | 4 +- .../runtime/taskexecutor/slot/TaskSlotUtils.java | 2 +- .../example/failing/JobSubmissionFailsITCase.java | 2 +- .../test/streaming/runtime/BackPressureITCase.java | 4 +- .../apache/flink/yarn/cli/FlinkYarnSessionCli.java | 2 +- .../apache/flink/yarn/YarnResourceManagerTest.java | 2 +- 36 files changed, 313 insertions(+), 313 deletions(-)
[flink] branch master updated (e17907a -> 7513d9b)
This is an automated email from the ASF dual-hosted git repository. azagrebin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from e17907a [hotfix][typo] Fix doc typo for the java doc of MultiStateKeyIteratorTest add 7513d9b [FLINK-15517][Runtime][Configuration][Network] Use back 'network' in 'shuffle' memory config option names No new revisions were added by this update. Summary of changes: .../task_manager_memory_configuration.html | 44 +++--- docs/ops/config.md | 6 +- docs/ops/config.zh.md | 6 +- docs/ops/deployment/yarn_setup.md | 2 +- docs/ops/deployment/yarn_setup.zh.md | 2 +- .../org/apache/flink/client/cli/ExecutorCLI.java | 2 +- .../flink/configuration/ConfigurationUtils.java| 4 +- .../NettyShuffleEnvironmentOptions.java| 10 +- .../flink/configuration/TaskManagerOptions.java| 40 +++--- flink-dist/src/main/resources/flink-conf.yaml | 6 +- .../test-scripts/test_batch_allround.sh| 4 +- .../test-scripts/test_heavy_deployment.sh | 4 +- .../test_high_parallelism_iterations.sh| 4 +- .../flink/kubernetes/KubernetesUtilsTest.java | 2 +- .../runtime/clusterframework/BootstrapTools.java | 2 +- .../clusterframework/TaskExecutorResourceSpec.java | 20 +-- .../TaskExecutorResourceUtils.java | 160 ++--- .../clusterframework/types/ResourceProfile.java| 50 +++ .../io/network/NettyShuffleServiceFactory.java | 2 +- .../io/network/buffer/NetworkBufferPool.java | 6 +- .../minicluster/MiniClusterConfiguration.java | 8 +- .../runtime/shuffle/ShuffleEnvironmentContext.java | 10 +- .../runtime/taskexecutor/TaskManagerServices.java | 2 +- .../TaskManagerServicesConfiguration.java | 4 +- .../NettyShuffleEnvironmentConfiguration.java | 14 +- .../clusterframework/BootstrapToolsTest.java | 2 +- .../TaskExecutorResourceUtilsTest.java | 142 +- .../types/ResourceProfileTest.java | 42 +++--- .../jobmaster/slotpool/SlotSharingManagerTest.java | 6 +- .../minicluster/MiniClusterConfigurationTest.java | 4 +- .../taskexecutor/TaskManagerRunnerStartupTest.java | 4 +- .../runtime/taskexecutor/slot/TaskSlotUtils.java | 2 +- .../example/failing/JobSubmissionFailsITCase.java | 2 +- .../test/streaming/runtime/BackPressureITCase.java | 4 +- .../apache/flink/yarn/cli/FlinkYarnSessionCli.java | 2 +- .../apache/flink/yarn/YarnResourceManagerTest.java | 2 +- 36 files changed, 313 insertions(+), 313 deletions(-)
[flink] branch master updated (4852002 -> e17907a)
This is an automated email from the ASF dual-hosted git repository. aljoscha pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 4852002 [FLINK-15327][runtime] No warning of InterruptedException during cancel. add e17907a [hotfix][typo] Fix doc typo for the java doc of MultiStateKeyIteratorTest No new revisions were added by this update. Summary of changes: .../org/apache/flink/state/api/input/MultiStateKeyIteratorTest.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[flink] branch release-1.10 updated (7c09f70 -> 8acd60b)
This is an automated email from the ASF dual-hosted git repository. pnowojski pushed a change to branch release-1.10 in repository https://gitbox.apache.org/repos/asf/flink.git. from 7c09f70 [FLINK-15554][azure] Bump jetty-util to 3.1.2 add 8acd60b [FLINK-15327][runtime] No warning of InterruptedException during cancel. No new revisions were added by this update. Summary of changes: .../flink/streaming/runtime/tasks/StreamTask.java | 29 ++-- .../runtime/tasks/SourceStreamTaskTest.java| 32 ++ 2 files changed, 30 insertions(+), 31 deletions(-)
[flink] branch master updated (a9411b7 -> 4852002)
This is an automated email from the ASF dual-hosted git repository. pnowojski pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from a9411b7 [FLINK-15554][azure] Bump jetty-util to 3.1.2 add 4852002 [FLINK-15327][runtime] No warning of InterruptedException during cancel. No new revisions were added by this update. Summary of changes: .../flink/streaming/runtime/tasks/StreamTask.java | 29 ++-- .../runtime/tasks/SourceStreamTaskTest.java| 32 ++ 2 files changed, 30 insertions(+), 31 deletions(-)
[flink] branch release-1.9 updated: [FLINK-15554][azure] Bump jetty-util to 3.1.2
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.9
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.9 by this push:
new 05eca9a [FLINK-15554][azure] Bump jetty-util to 3.1.2
05eca9a is described below
commit 05eca9a8ca1388e3f20f1a72d114c19afc54a01d
Author: Chesnay Schepler
AuthorDate: Wed Jan 15 11:58:50 2020 +0100
[FLINK-15554][azure] Bump jetty-util to 3.1.2
---
flink-filesystems/flink-azure-fs-hadoop/pom.xml | 12
.../flink-azure-fs-hadoop/src/main/resources/META-INF/NOTICE | 4 ++--
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/flink-filesystems/flink-azure-fs-hadoop/pom.xml
b/flink-filesystems/flink-azure-fs-hadoop/pom.xml
index 781e851..b6c8fd2 100644
--- a/flink-filesystems/flink-azure-fs-hadoop/pom.xml
+++ b/flink-filesystems/flink-azure-fs-hadoop/pom.xml
@@ -36,6 +36,7 @@ under the License.
1.16.0
2.9.4
+ 9.3.24.v20180605
@@ -90,6 +91,17 @@ under the License.
+
+
+
+
+ org.eclipse.jetty
+ jetty-util-ajax
+ ${jetty.version}
+
+
+
+
diff --git
a/flink-filesystems/flink-azure-fs-hadoop/src/main/resources/META-INF/NOTICE
b/flink-filesystems/flink-azure-fs-hadoop/src/main/resources/META-INF/NOTICE
index 9235626..5aeadd3 100644
--- a/flink-filesystems/flink-azure-fs-hadoop/src/main/resources/META-INF/NOTICE
+++ b/flink-filesystems/flink-azure-fs-hadoop/src/main/resources/META-INF/NOTICE
@@ -17,5 +17,5 @@ This project bundles the following dependencies under the
Apache Software Licens
- org.apache.hadoop:hadoop-azure:3.1.0
- org.apache.httpcomponents:httpclient:4.5.3
- org.apache.httpcomponents:httpcore:4.4.6
-- org.eclipse.jetty:jetty-util:9.3.19.v20170502
-- org.eclipse.jetty:jetty-util-ajax:9.3.19.v20170502
+- org.eclipse.jetty:jetty-util:9.3.24.v20180605
+- org.eclipse.jetty:jetty-util-ajax:9.3.24.v20180605
[flink] branch release-1.10 updated (8927e97 -> 7c09f70)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch release-1.10 in repository https://gitbox.apache.org/repos/asf/flink.git. from 8927e97 [FLINK-15535][doc] Add ProcessFunctionTestHarnesses documentation add 7c09f70 [FLINK-15554][azure] Bump jetty-util to 3.1.2 No new revisions were added by this update. Summary of changes: flink-filesystems/flink-azure-fs-hadoop/pom.xml | 12 .../flink-azure-fs-hadoop/src/main/resources/META-INF/NOTICE | 4 ++-- 2 files changed, 14 insertions(+), 2 deletions(-)
[flink] branch master updated (d7edaaa -> a9411b7)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from d7edaaa [FLINK-15535][doc] Add ProcessFunctionTestHarnesses documentation add a9411b7 [FLINK-15554][azure] Bump jetty-util to 3.1.2 No new revisions were added by this update. Summary of changes: flink-filesystems/flink-azure-fs-hadoop/pom.xml | 12 .../flink-azure-fs-hadoop/src/main/resources/META-INF/NOTICE | 4 ++-- 2 files changed, 14 insertions(+), 2 deletions(-)
[flink] branch release-1.10 updated (1ec6f99 -> 8927e97)
This is an automated email from the ASF dual-hosted git repository. kkloudas pushed a change to branch release-1.10 in repository https://gitbox.apache.org/repos/asf/flink.git. from 1ec6f99 [FLINK-15278] Update the StreamingFileSink docs add 8927e97 [FLINK-15535][doc] Add ProcessFunctionTestHarnesses documentation No new revisions were added by this update. Summary of changes: docs/dev/stream/testing.md| 84 +++ docs/dev/stream/testing.zh.md | 84 +++ 2 files changed, 168 insertions(+)
[flink] branch master updated (cec75ef -> d7edaaa)
This is an automated email from the ASF dual-hosted git repository. kkloudas pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from cec75ef [FLINK-15278] Update the StreamingFileSink docs add d7edaaa [FLINK-15535][doc] Add ProcessFunctionTestHarnesses documentation No new revisions were added by this update. Summary of changes: docs/dev/stream/testing.md| 84 +++ docs/dev/stream/testing.zh.md | 84 +++ 2 files changed, 168 insertions(+)
[flink] branch release-1.10 updated (efe266f -> 1ec6f99)
This is an automated email from the ASF dual-hosted git repository. kkloudas pushed a change to branch release-1.10 in repository https://gitbox.apache.org/repos/asf/flink.git. from efe266f [FLINK-15434][tests] Harden JobMasterTest add 1ec6f99 [FLINK-15278] Update the StreamingFileSink docs No new revisions were added by this update. Summary of changes: docs/dev/connectors/streamfile_sink.md | 288 - 1 file changed, 180 insertions(+), 108 deletions(-)
[flink] branch master updated (e1ad65b -> cec75ef)
This is an automated email from the ASF dual-hosted git repository. kkloudas pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from e1ad65b [FLINK-15434][tests] Harden JobMasterTest add cec75ef [FLINK-15278] Update the StreamingFileSink docs No new revisions were added by this update. Summary of changes: docs/dev/connectors/streamfile_sink.md | 288 - 1 file changed, 180 insertions(+), 108 deletions(-)
[flink] branch release-1.10 updated (4b78a4e -> efe266f)
This is an automated email from the ASF dual-hosted git repository. gary pushed a change to branch release-1.10 in repository https://gitbox.apache.org/repos/asf/flink.git. from 4b78a4e [FLINK-14091] Harden ZKCheckpointIDCounterMultiServersTest add efe266f [FLINK-15434][tests] Harden JobMasterTest No new revisions were added by this update. Summary of changes: .../test/java/org/apache/flink/runtime/jobmaster/JobMasterTest.java | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-)
[flink] branch master updated (7455a09 -> e1ad65b)
This is an automated email from the ASF dual-hosted git repository. gary pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 7455a09 [FLINK-14091] Harden ZKCheckpointIDCounterMultiServersTest add e1ad65b [FLINK-15434][tests] Harden JobMasterTest No new revisions were added by this update. Summary of changes: .../test/java/org/apache/flink/runtime/jobmaster/JobMasterTest.java | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-)
[flink] branch release-1.10 updated (7667d43 -> 4b78a4e)
This is an automated email from the ASF dual-hosted git repository.
trohrmann pushed a change to branch release-1.10
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 7667d43 [FLINK-15576] remove isTemporary property from
CatalogFunction API
add 7181254 [FLINK-14091][coordination] Allow updates to connection state
when ZKCheckpointIDCounter reconnects to ZK
add 7f27bb6 [FLINK-14091][tests] Tests ZooKeeperCheckpointIDCounter can
be recoverd from connection loss
add 7fcda36 [FLINK-14091][tests] Refactor ZooKeeperCheckpointIDCounter
for a more testable codebase
add 361de78 [hotfix] Fix checkstyle violations in
ZooKeeperCheckpointIDCounter
add 4b78a4e [FLINK-14091] Harden ZKCheckpointIDCounterMultiServersTest
No new revisions were added by this update.
Summary of changes:
.../DefaultLastStateConnectionStateListener.java} | 24 +++--
.../LastStateConnectionStateListener.java} | 21 ++--
.../checkpoint/ZooKeeperCheckpointIDCounter.java | 59 ---
.../apache/flink/runtime/util/ZooKeeperUtils.java | 3 +-
.../checkpoint/CheckpointIDCounterTest.java| 6 +-
.../ZKCheckpointIDCounterMultiServersTest.java | 118 +
.../flink/runtime/zookeeper/ZooKeeperResource.java | 5 +
.../zookeeper/ZooKeeperTestEnvironment.java| 16 ++-
8 files changed, 189 insertions(+), 63 deletions(-)
copy
flink-runtime/src/{test/java/org/apache/flink/runtime/checkpoint/NoOpFailJobCall.java
=>
main/java/org/apache/flink/runtime/checkpoint/DefaultLastStateConnectionStateListener.java}
(61%)
copy
flink-runtime/src/{test/java/org/apache/flink/runtime/checkpoint/NoOpFailJobCall.java
=>
main/java/org/apache/flink/runtime/checkpoint/LastStateConnectionStateListener.java}
(62%)
create mode 100644
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZKCheckpointIDCounterMultiServersTest.java
[flink] branch master updated (8d8e708 -> 7455a09)
This is an automated email from the ASF dual-hosted git repository.
trohrmann pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 8d8e708 [FLINK-15431][CEP] Add numLateRecordsDropped metric in
CepOperator
add 4b98956 [FLINK-14091][coordination] Allow updates to connection state
when ZKCheckpointIDCounter reconnects to ZK
add 25b1697 [FLINK-14091][tests] Tests ZooKeeperCheckpointIDCounter can
be recoverd from connection loss
add 7a0fa1e [FLINK-14091][tests] Refactor ZooKeeperCheckpointIDCounter
for a more testable codebase
add 75ecfae [hotfix] Fix checkstyle violations in
ZooKeeperCheckpointIDCounter
add 7455a09 [FLINK-14091] Harden ZKCheckpointIDCounterMultiServersTest
No new revisions were added by this update.
Summary of changes:
.../DefaultLastStateConnectionStateListener.java} | 24 +++--
.../LastStateConnectionStateListener.java} | 21 ++--
.../checkpoint/ZooKeeperCheckpointIDCounter.java | 59 ---
.../apache/flink/runtime/util/ZooKeeperUtils.java | 3 +-
.../checkpoint/CheckpointIDCounterTest.java| 6 +-
.../ZKCheckpointIDCounterMultiServersTest.java | 118 +
.../flink/runtime/zookeeper/ZooKeeperResource.java | 5 +
.../zookeeper/ZooKeeperTestEnvironment.java| 16 ++-
8 files changed, 189 insertions(+), 63 deletions(-)
copy
flink-runtime/src/{test/java/org/apache/flink/runtime/checkpoint/NoOpFailJobCall.java
=>
main/java/org/apache/flink/runtime/checkpoint/DefaultLastStateConnectionStateListener.java}
(61%)
copy
flink-runtime/src/{test/java/org/apache/flink/runtime/checkpoint/NoOpFailJobCall.java
=>
main/java/org/apache/flink/runtime/checkpoint/LastStateConnectionStateListener.java}
(62%)
create mode 100644
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZKCheckpointIDCounterMultiServersTest.java
