This is an automated email from the ASF dual-hosted git repository.
martijnvisser pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 291ad90f3 Rebuild website
291ad90f3 is described below
commit 291ad90f38367066c2a8c0a278b04a5b404dc0d0
Author: MartijnVisser <[email protected]>
AuthorDate: Wed May 18 09:46:19 2022 +0200
Rebuild website
---
content/2022/05/18/latency-part1.html | 369 ++++++++++++++++++++++++++++++++++
content/blog/feed.xml | 248 ++++++++++-------------
content/blog/index.html | 39 ++--
content/blog/page10/index.html | 38 ++--
content/blog/page11/index.html | 38 ++--
content/blog/page12/index.html | 38 ++--
content/blog/page13/index.html | 40 ++--
content/blog/page14/index.html | 40 ++--
content/blog/page15/index.html | 40 ++--
content/blog/page16/index.html | 38 ++--
content/blog/page17/index.html | 36 ++--
content/blog/page18/index.html | 38 ++--
content/blog/page19/index.html | 25 +++
content/blog/page2/index.html | 39 ++--
content/blog/page3/index.html | 38 ++--
content/blog/page4/index.html | 38 ++--
content/blog/page5/index.html | 38 ++--
content/blog/page6/index.html | 38 ++--
content/blog/page7/index.html | 40 ++--
content/blog/page8/index.html | 43 ++--
content/blog/page9/index.html | 39 ++--
content/index.html | 12 +-
content/zh/index.html | 12 +-
23 files changed, 939 insertions(+), 425 deletions(-)
diff --git a/content/2022/05/18/latency-part1.html
b/content/2022/05/18/latency-part1.html
new file mode 100644
index 000000000..f67b31407
--- /dev/null
+++ b/content/2022/05/18/latency-part1.html
@@ -0,0 +1,369 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head
content must come *after* these tags -->
+ <title>Apache Flink: Getting into Low-Latency Gears with Apache Flink -
Part One</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml"
title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics
event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media
queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script
src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script>
+ <script
src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script>
+ <![endif]-->
+ <!-- Matomo -->
+ <script>
+ var _paq = window._paq = window._paq || [];
+ /* tracker methods like "setCustomDimension" should be called before
"trackPageView" */
+ /* We explicitly disable cookie tracking to avoid privacy issues */
+ _paq.push(['disableCookies']);
+ /* Measure a visit to flink.apache.org and nightlies.apache.org/flink as
the same visit */
+ _paq.push(["setDomains",
["*.flink.apache.org","*.nightlies.apache.org/flink"]]);
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//matomo.privacy.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '1']);
+ var d=document, g=d.createElement('script'),
s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+ <!-- End Matomo Code -->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed"
data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg"
width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse"
id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream
Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache
Flink?</a></li>
+
+
+
+ <!-- Stateful Functions? -->
+
+ <li><a
href="https://nightlies.apache.org/flink/flink-statefun-docs-stable/";>What is
Stateful Functions?</a></li>
+
+ <!-- Flink ML? -->
+
+ <li><a
href="https://nightlies.apache.org/flink/flink-ml-docs-stable/";>What is Flink
ML?</a></li>
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Getting Started -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown"
href="#">Getting Started<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a
href="https://nightlies.apache.org/flink/flink-docs-release-1.15//docs/try-flink/local_installation/";
target="_blank">With Flink <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-statefun-docs-release-3.2/getting-started/project-setup.html";
target="_blank">With Flink Stateful Functions <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-ml-docs-release-2.0/try-flink-ml/quick-start.html";
target="_blank">With Flink ML <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-0.1/try-flink-kubernetes-operator/quick-start.html";
target="_blank">With Flink Kubernetes Operator <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-table-store-docs-release-0.1/try-table-store/quick-start.html";
target="_blank">With Flink Table Store <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a href="/training.html">Training Course</a></li>
+ </ul>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown"
href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a
href="https://nightlies.apache.org/flink/flink-docs-release-1.15";
target="_blank">Flink 1.15 (Latest stable release) <small><span
class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-docs-master";
target="_blank">Flink Master (Latest Snapshot) <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-statefun-docs-release-3.2";
target="_blank">Flink Stateful Functions 3.2 (Latest stable release)
<small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-statefun-docs-master";
target="_blank">Flink Stateful Functions Master (Latest Snapshot) <small><span
class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-ml-docs-release-2.0";
target="_blank">Flink ML 2.0 (Latest stable release) <small><span
class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-ml-docs-master";
target="_blank">Flink ML Master (Latest Snapshot) <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-0.1";
target="_blank">Flink Kubernetes Operator 0.1 (Latest stable release)
<small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main";
target="_blank">Flink Kubernetes Operator Main (Latest Snapshot) <small><span
class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-table-store-docs-release-0.1";
target="_blank">Flink Table Store 0.1 (Latest stable release) <small><span
class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://nightlies.apache.org/flink/flink-table-store-docs-master";
target="_blank">Flink Table Store Master (Latest Snapshot) <small><span
class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+ <!-- Flink-packages -->
+ <li>
+ <a href="https://flink-packages.org";
target="_blank">flink-packages.org <small><span class="glyphicon
glyphicon-new-window"></span></small></a>
+ </li>
+
+
+ <!-- Third menu section aim to support community and contributors
-->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to
Contribute</a></li>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink"; target="_blank">Flink
on GitHub <small><span class="glyphicon
glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <a href="/zh/2022/05/18/latency-part1.html">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <style>
+ .smalllinks:link {
+ display: inline-block !important; background: none; padding-top:
0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px;
+ }
+ </style>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink";
target="_blank">@ApacheFlink <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/"
target="_blank">Plan Visualizer <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+
+ <li >
+ <a href="/security.html">Flink Security</a>
+ </li>
+
+ <hr />
+
+ <li><a href="https://apache.org"; target="_blank">Apache Software
Foundation <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+
+ <li>
+
+ <a class="smalllinks" href="https://www.apache.org/licenses/";
target="_blank">License</a> <small><span class="glyphicon
glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/security/";
target="_blank">Security</a> <small><span class="glyphicon
glyphicon-new-window"></span></small>
+
+ <a class="smalllinks"
href="https://www.apache.org/foundation/sponsorship.html";
target="_blank">Donate</a> <small><span class="glyphicon
glyphicon-new-window"></span></small>
+
+ <a class="smalllinks"
href="https://www.apache.org/foundation/thanks.html"; target="_blank">Thanks</a>
<small><span class="glyphicon glyphicon-new-window"></span></small>
+ </li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <div class="row">
+ <h1>Getting into Low-Latency Gears with Apache Flink - Part One</h1>
+ <p><i></i></p>
+
+ <article>
+ <p>18 May 2022 Jun Qin & Nico Kruber </p>
+
+<p>Apache Flink is a stream processing framework well known for its low
latency processing capabilities. It is generic and suitable for a wide range of
use cases. As a Flink application developer or a cluster administrator, you
need to find the right gear that is best for your application. In other words,
you don’t want to be driving a luxury sports car while only using the first
gear.</p>
+
+<p>In this multi-part series, we will present a collection of low-latency
techniques in Flink. Part one starts with types of latency in Flink and the way
we measure the end-to-end latency, followed by a few techniques that optimize
latency directly. Part two continues with a few more direct latency
optimization techniques. Further parts of this series will cover techniques
that improve latencies by optimizing throughput. For each optimization
technique, we will clarify what it is, when t [...]
+
+<p>This series of blog posts is a write-up of <a
href="https://www.youtube.com/watch?v=4dwwokhQHwo";>our talk in Flink Forward
Global 2021</a> and includes additional latency optimization techniques and
details.</p>
+
+<h1 id="latency">Latency</h1>
+
+<h2 id="types-of-latency">Types of latency</h2>
+
+<p>Latency can refer to different things. <a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/ops/metrics/#end-to-end-latency-tracking";>LatencyMarkers</a>
in Flink measure the time it takes for the markers to travel from each source
operator to each downstream operator. As LatencyMarkers bypass user functions
in operators, the measured latencies do not reflect the entire end-to-end
latency but only a part of it. Flink also supports tracking the <a
href="https://nightlies.a [...]
+
+<h2 id="how-we-measure-end-to-end-latency">How we measure end-to-end
latency</h2>
+
+<p>There are two scenarios to consider. In the first scenario, a pipeline does
a simple transformation, and there are no timers or any other complex event
time logic. For example, a pipeline that produces one output event for each
input event. In this case, we measure the processing delay as the latency, that
is, <code>t2 - t1</code> as shown in the diagram.</p>
+
+<center>
+<img vspace="8" style="width:40%"
src="/img/blog/2022-05-18-latency-part1/scenario1-simple.png" />
+</center>
+
+<p>The second scenario is where complex event time logic is involved (e.g.,
timers, aggregation, windowing). In this case, we measure the event time lag as
the latency, that is, <code>current processing time - current watermark</code>.
The event time lag gives us the difference between the expected output time and
the actual output time.</p>
+
+<center>
+<img vspace="8" style="width:40%"
src="/img/blog/2022-05-18-latency-part1/scenario2-eventtime.png" />
+</center>
+
+<p>In both scenarios, we capture a histogram and show the 99th percentile of
the end-to-end latency. The latency we measure here includes the time an event
stays in the source message queue (e.g., Kafka). The reason for this is that it
covers the scenarios where a source operator in a pipeline is backpressured by
other operators. The more the source operator is backpressured, the longer the
messages stay in the message queue. So, including the time events stay in the
message queue in the [...]
+
+<h1 id="low-latency-optimization-techniques">Low-latency optimization
techniques</h1>
+
+<p>We will discuss low-latency techniques in two groups: techniques that
optimize latency directly and techniques that improve latency by optimizing
throughput.
+Each of these techniques can be as simple as a configuration change or may
require code changes, or both. We have created a git repository containing the
example jobs used in our experiments to support our statements. Keep in mind
that all the experimental results we will show are specific to those jobs and
the environment they run in. Your job may show different results depending on
where the latency bottleneck is.</p>
+
+<h2 id="direct-latency-optimization">Direct latency optimization</h2>
+
+<h3 id="allocate-enough-resources">Allocate enough resources</h3>
+<p>An obvious but often forgotten low-latency technique is to allocate enough
resources to your job. Flink has some metrics (e.g., <a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/ops/monitoring/back_pressure/#task-performance-metrics";>idleTimeMsPerSecond</a>,
<a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/ops/monitoring/back_pressure/#task-performance-metrics";>busyTimeMsPerSecond</a>,
<a href="https://nightlies.apache.org/flink/flink-docs-stable/doc [...]
+
+<p><strong>You can apply this optimization</strong> if your job or part of it
is running at its total CPU/memory capacity and you have more resources that
can be allocated to the job. In the case of scaling out with high parallelism,
your streaming job must be able to make use of the additional resources. For
example, the job should not have fixed parallelisms in the code, the job should
not be bottlenecked on the source streams, and the input streams are
partitionable by keys such that [...]
+
+<p><strong>Keep in mind</strong> that allocating more resources may result in
increased financial costs, especially when you are running jobs in the
cloud.</p>
+
+<p>Below are the experimental results of <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java";>WindowingJob</a>.
As you can see from the graph at the left, when the parallelism was 2, the two
subtasks were often 100% busy. After we increased the parallelism to 3, the
three subtasks were around 75% busy. As a result, the 99th percentile latency
reduces from around 3 seconds to 650 milliseconds.</p>
+
+<center>
+<img vspace="8" style="width:90%"
src="/img/blog/2022-05-18-latency-part1/increase-parallelism.png" />
+</center>
+
+<h3 id="use-applicable-state-backends">Use applicable state backends</h3>
+
+<p>When using the <code>filesystem</code> (Flink 1.12 or early) or
<code>hashmap</code> (Flink 1.13 or later) state backend, the state objects are
stored in memory and can be accessed directly. In contrast, when using the
<code>rocksdb</code> state backend, every state access has to go through a
(de-)serialization process which in addition may involve disk accesses. So
using <code>filesystem/hashmap</code> state backend can help reduce latency.</p>
+
+<p><strong>You can apply this optimization</strong> if your state size is very
small compared to the memory you can allocate to your job and your state size
will not grow beyond your memory capacity. You can set the <a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/deployment/memory/mem_setup_tm/#managed-memory";>managed
memory</a> size to 0 if not needed. Since Flink 1.13, you can always start
with the <code>hashmap</code> state backend and seamlessly switch to the <code
[...]
+
+<p><strong>Keep in mind</strong> that heap-based state backends use more
memory compared with RocksDB due to their copy-on-write data structure and
Java’s on-heap object representation. Heap-based state backends can be affected
by the garbage collector which makes them less predictable and may lead to high
tail latencies. Also, as of now, there is no support for incremental
checkpointing (this is being developed in <a
href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-151%3A+In [...]
+
+<p>Our experiments with the previously mentioned <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java";>WindowingJob</a>
after switching the state backend from <code>rocksdb</code> to
<code>hashmap</code> show a further reduction of the latency down to 500ms.
Depending on your job’s state access pattern, you may see larger or smaller
improvements. The graph on the right shows the garbage collection’s impact on
the l [...]
+
+<center>
+<img vspace="8" style="width:90%"
src="/img/blog/2022-05-18-latency-part1/choose-state-backend.png" />
+</center>
+
+<h3 id="emit-watermarks-quickly">Emit watermarks quickly</h3>
+
+<p>When using a periodic <a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/datastream/event-time/generating_watermarks/";>watermark
generator</a>, Flink generates a watermark every 200 ms. This means that, by
default, each parallel watermark generator does not produce watermark updates
until 200 ms have passed. While this may be sufficient for many cases, if you
are aiming for sub-second latencies, you could try reducing the interval even
further, for example, to 100 ms.</p>
+
+<p><strong>You can apply this optimization</strong> if you use event time and
a periodic watermark generator, and you are aiming for sub-second latencies.</p>
+
+<p><strong>Keep in mind</strong> that watermark generation that is too
frequent may also degrade performance because more watermarks must be processed
by the framework. Moreover, even though watermarks are only created every 200
milliseconds, watermarks may arrive at much higher frequencies further
downstream in your job because tasks may receive watermarks from multiple
parallel watermark generators.</p>
+
+<p>We re-ran the previous <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java";>WindowingJob</a>
experiment with the reduced watermark interval
<code>pipeline.auto-watermark-interval: 100ms</code> and reduced the latency
further to 430ms.</p>
+
+<center>
+<img vspace="8" style="width:50%"
src="/img/blog/2022-05-18-latency-part1/watermark-interval.png" />
+</center>
+
+<h3 id="flush-network-buffers-early">Flush network buffers early</h3>
+
+<p>Flink uses buffers when sending data from one task to another over the
network. Buffers are flushed and sent out when they are filled up or when the
default timeout of 100ms has passed. Again, if you are aiming for sub-second
latencies, you can lower the timeout to reduce latencies.</p>
+
+<p><strong>You can apply this optimization</strong> if you are aiming for
sub-second latencies.</p>
+
+<p><strong>Keep in mind</strong> that network buffer timeout that is too low
may reduce throughput.</p>
+
+<p>As seen in the following experiment results, by using
<code>execution.buffer-timeout: 10 ms</code> in <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java";>WindowingJob</a>,
we again reduced the latency (now to 370ms).</p>
+
+<center>
+<img vspace="8" style="width:50%"
src="/img/blog/2022-05-18-latency-part1/buffer-timeout.png" />
+</center>
+
+<h1 id="summary">Summary</h1>
+
+<p>In part one of this multi-part series, we discussed types of latency in
Flink and the way we measure end-to-end latency. Then we presented a few
latency optimization techniques with a focus on direct latency optimization.
For each technique, we explained what it is, when to use it, and what to keep
in mind when using it. Part two will continue with a few more direct latency
optimization techniques. Stay tuned!</p>
+
+ </article>
+ </div>
+
+ <div class="row">
+ <div id="disqus_thread"></div>
+ <script type="text/javascript">
+ /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE
* * */
+ var disqus_shortname = 'stratosphere-eu'; // required: replace example
with your forum shortname
+
+ /* * * DON'T EDIT BELOW THIS LINE * * */
+ (function() {
+ var dsq = document.createElement('script'); dsq.type =
'text/javascript'; dsq.async = true;
+ dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+ (document.getElementsByTagName('head')[0] ||
document.getElementsByTagName('body')[0]).appendChild(dsq);
+ })();
+ </script>
+ </div>
+ </div>
+</div>
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2022 <a href="http://apache.org";>The Apache
Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache
feather logo are either registered trademarks or trademarks of The Apache
Software Foundation.</p>
+ <p><a
href="https://privacy.apache.org/policies/privacy-policy-public.html";>Privacy
Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as
needed -->
+ <script src="/js/jquery.matchHeight-min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+ </body>
+</html>
diff --git a/content/blog/feed.xml b/content/blog/feed.xml
index 38ef3a58e..91ed5cbed 100644
--- a/content/blog/feed.xml
+++ b/content/blog/feed.xml
@@ -6,6 +6,107 @@
<link>https://flink.apache.org/blog</link>
<atom:link href="https://flink.apache.org/blog/feed.xml"; rel="self"
type="application/rss+xml" />
+<item>
+<title>Getting into Low-Latency Gears with Apache Flink - Part One</title>
+<description><p>Apache Flink is a stream processing framework well known
for its low latency processing capabilities. It is generic and suitable for a
wide range of use cases. As a Flink application developer or a cluster
administrator, you need to find the right gear that is best for your
application. In other words, you don’t want to be driving a luxury sports car
while only using the first gear.</p>
+
+<p>In this multi-part series, we will present a collection of
low-latency techniques in Flink. Part one starts with types of latency in Flink
and the way we measure the end-to-end latency, followed by a few techniques
that optimize latency directly. Part two continues with a few more direct
latency optimization techniques. Further parts of this series will cover
techniques that improve latencies by optimizing throughput. For each
optimization technique, we will clarify what it is, [...]
+
+<p>This series of blog posts is a write-up of <a
href="https://www.youtube.com/watch?v=4dwwokhQHwo">our talk in
Flink Forward Global 2021</a> and includes additional latency
optimization techniques and details.</p>
+
+<h1 id="latency">Latency</h1>
+
+<h2 id="types-of-latency">Types of latency</h2>
+
+<p>Latency can refer to different things. <a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/ops/metrics/#end-to-end-latency-tracking">LatencyMarkers</a>
in Flink measure the time it takes for the markers to travel from each source
operator to each downstream operator. As LatencyMarkers bypass user functions
in operators, the measured latencies do not reflect the entire end-to-end
latency but only a part of it. Flink also supports tracking the [...]
+
+<h2 id="how-we-measure-end-to-end-latency">How we measure
end-to-end latency</h2>
+
+<p>There are two scenarios to consider. In the first scenario, a
pipeline does a simple transformation, and there are no timers or any other
complex event time logic. For example, a pipeline that produces one output
event for each input event. In this case, we measure the processing delay as
the latency, that is, <code>t2 - t1</code> as shown in the
diagram.</p>
+
+<center>
+<img vspace="8" style="width:40%"
src="/img/blog/2022-05-18-latency-part1/scenario1-simple.png" />
+</center>
+
+<p>The second scenario is where complex event time logic is involved
(e.g., timers, aggregation, windowing). In this case, we measure the event time
lag as the latency, that is, <code>current processing time - current
watermark</code>. The event time lag gives us the difference between the
expected output time and the actual output time.</p>
+
+<center>
+<img vspace="8" style="width:40%"
src="/img/blog/2022-05-18-latency-part1/scenario2-eventtime.png" />
+</center>
+
+<p>In both scenarios, we capture a histogram and show the 99th
percentile of the end-to-end latency. The latency we measure here includes the
time an event stays in the source message queue (e.g., Kafka). The reason for
this is that it covers the scenarios where a source operator in a pipeline is
backpressured by other operators. The more the source operator is
backpressured, the longer the messages stay in the message queue. So, including
the time events stay in the message queue [...]
+
+<h1 id="low-latency-optimization-techniques">Low-latency
optimization techniques</h1>
+
+<p>We will discuss low-latency techniques in two groups: techniques that
optimize latency directly and techniques that improve latency by optimizing
throughput.
+Each of these techniques can be as simple as a configuration change or may
require code changes, or both. We have created a git repository containing the
example jobs used in our experiments to support our statements. Keep in mind
that all the experimental results we will show are specific to those jobs and
the environment they run in. Your job may show different results depending on
where the latency bottleneck is.</p>
+
+<h2 id="direct-latency-optimization">Direct latency
optimization</h2>
+
+<h3 id="allocate-enough-resources">Allocate enough
resources</h3>
+<p>An obvious but often forgotten low-latency technique is to allocate
enough resources to your job. Flink has some metrics (e.g., <a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/ops/monitoring/back_pressure/#task-performance-metrics">idleTimeMsPerSecond</a>,
<a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/ops/monitoring/back_pressure/#task-performance-metrics">busyTimeMsPerSecond</a>,
<a href=&quo [...]
+
+<p><strong>You can apply this optimization</strong> if your
job or part of it is running at its total CPU/memory capacity and you have more
resources that can be allocated to the job. In the case of scaling out with
high parallelism, your streaming job must be able to make use of the additional
resources. For example, the job should not have fixed parallelisms in the code,
the job should not be bottlenecked on the source streams, and the input streams
are partitionable [...]
+
+<p><strong>Keep in mind</strong> that allocating more
resources may result in increased financial costs, especially when you are
running jobs in the cloud.</p>
+
+<p>Below are the experimental results of <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java">WindowingJob</a>.
As you can see from the graph at the left, when the parallelism was 2, the two
subtasks were often 100% busy. After we increased the parallelism to 3, the
three subtasks were around 75% busy. As a result, the 99th percentile latency
reduces from around 3 seconds to 650 millise [...]
+
+<center>
+<img vspace="8" style="width:90%"
src="/img/blog/2022-05-18-latency-part1/increase-parallelism.png"
/>
+</center>
+
+<h3 id="use-applicable-state-backends">Use applicable state
backends</h3>
+
+<p>When using the <code>filesystem</code> (Flink 1.12 or
early) or <code>hashmap</code> (Flink 1.13 or later) state backend,
the state objects are stored in memory and can be accessed directly. In
contrast, when using the <code>rocksdb</code> state backend, every
state access has to go through a (de-)serialization process which in addition
may involve disk accesses. So using <code>filesystem/hashmap</code>
state backend can help reduc [...]
+
+<p><strong>You can apply this optimization</strong> if your
state size is very small compared to the memory you can allocate to your job
and your state size will not grow beyond your memory capacity. You can set the
<a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/deployment/memory/mem_setup_tm/#managed-memory">managed
memory</a> size to 0 if not needed. Since Flink 1.13, you can always
start with the <code>hashmap</code& [...]
+
+<p><strong>Keep in mind</strong> that heap-based state
backends use more memory compared with RocksDB due to their copy-on-write data
structure and Java’s on-heap object representation. Heap-based state backends
can be affected by the garbage collector which makes them less predictable and
may lead to high tail latencies. Also, as of now, there is no support for
incremental checkpointing (this is being developed in <a
href="https://cwiki.apache.org/confluence/di [...]
+
+<p>Our experiments with the previously mentioned <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java">WindowingJob</a>
after switching the state backend from <code>rocksdb</code> to
<code>hashmap</code> show a further reduction of the latency down
to 500ms. Depending on your job’s state access pattern, you may see larger or
smaller improvements. The graph on the [...]
+
+<center>
+<img vspace="8" style="width:90%"
src="/img/blog/2022-05-18-latency-part1/choose-state-backend.png"
/>
+</center>
+
+<h3 id="emit-watermarks-quickly">Emit watermarks
quickly</h3>
+
+<p>When using a periodic <a
href="https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/datastream/event-time/generating_watermarks/">watermark
generator</a>, Flink generates a watermark every 200 ms. This means
that, by default, each parallel watermark generator does not produce watermark
updates until 200 ms have passed. While this may be sufficient for many cases,
if you are aiming for sub-second latencies, you could try reducing the interval
even f [...]
+
+<p><strong>You can apply this optimization</strong> if you
use event time and a periodic watermark generator, and you are aiming for
sub-second latencies.</p>
+
+<p><strong>Keep in mind</strong> that watermark generation
that is too frequent may also degrade performance because more watermarks must
be processed by the framework. Moreover, even though watermarks are only
created every 200 milliseconds, watermarks may arrive at much higher
frequencies further downstream in your job because tasks may receive watermarks
from multiple parallel watermark generators.</p>
+
+<p>We re-ran the previous <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java">WindowingJob</a>
experiment with the reduced watermark interval
<code>pipeline.auto-watermark-interval: 100ms</code> and reduced
the latency further to 430ms.</p>
+
+<center>
+<img vspace="8" style="width:50%"
src="/img/blog/2022-05-18-latency-part1/watermark-interval.png" />
+</center>
+
+<h3 id="flush-network-buffers-early">Flush network buffers
early</h3>
+
+<p>Flink uses buffers when sending data from one task to another over
the network. Buffers are flushed and sent out when they are filled up or when
the default timeout of 100ms has passed. Again, if you are aiming for
sub-second latencies, you can lower the timeout to reduce latencies.</p>
+
+<p><strong>You can apply this optimization</strong> if you
are aiming for sub-second latencies.</p>
+
+<p><strong>Keep in mind</strong> that network buffer timeout
that is too low may reduce throughput.</p>
+
+<p>As seen in the following experiment results, by using
<code>execution.buffer-timeout: 10 ms</code> in <a
href="https://github.com/ververica/lab-flink-latency/blob/main/src/main/java/com/ververica/lablatency/job/WindowingJob.java">WindowingJob</a>,
we again reduced the latency (now to 370ms).</p>
+
+<center>
+<img vspace="8" style="width:50%"
src="/img/blog/2022-05-18-latency-part1/buffer-timeout.png" />
+</center>
+
+<h1 id="summary">Summary</h1>
+
+<p>In part one of this multi-part series, we discussed types of latency
in Flink and the way we measure end-to-end latency. Then we presented a few
latency optimization techniques with a focus on direct latency optimization.
For each technique, we explained what it is, when to use it, and what to keep
in mind when using it. Part two will continue with a few more direct latency
optimization techniques. Stay tuned!</p>
+</description>
+<pubDate>Wed, 18 May 2022 02:00:00 +0200</pubDate>
+<link>https://flink.apache.org/2022/05/18/latency-part1.html</link>
+<guid isPermaLink="true">/2022/05/18/latency-part1.html</guid>
+</item>
+
<item>
<title>Apache Flink Table Store 0.1.0 Release Announcement</title>
<description><p>The Apache Flink community is pleased to announce the
preview release of the
@@ -20241,152 +20342,5 @@ Enabling latency metrics can significantly impact the
performance of the cluster
<guid isPermaLink="true">/2019/07/23/flink-network-stack-2.html</guid>
</item>
-<item>
-<title>Apache Flink 1.8.1 Released</title>
-<description><p>The Apache Flink community released the first bugfix
version of the Apache Flink 1.8 series.</p>
-
-<p>This release includes more than 40 fixes and minor improvements for
Flink 1.8.1. The list below includes a detailed list of all improvements,
sub-tasks and bug fixes.</p>
-
-<p>We highly recommend all users to upgrade to Flink 1.8.1.</p>
-
-<p>Updated Maven dependencies:</p>
-
-<div class="highlight"><pre><code
class="language-xml"><span
class="nt">&lt;dependency&gt;</span>
- <span
class="nt">&lt;groupId&gt;</span>org.apache.flink<span
class="nt">&lt;/groupId&gt;</span>
- <span
class="nt">&lt;artifactId&gt;</span>flink-java<span
class="nt">&lt;/artifactId&gt;</span>
- <span
class="nt">&lt;version&gt;</span>1.8.1<span
class="nt">&lt;/version&gt;</span>
-<span class="nt">&lt;/dependency&gt;</span>
-<span class="nt">&lt;dependency&gt;</span>
- <span
class="nt">&lt;groupId&gt;</span>org.apache.flink<span
class="nt">&lt;/groupId&gt;</span>
- <span
class="nt">&lt;artifactId&gt;</span>flink-streaming-java_2.11<span
class="nt">&lt;/artifactId&gt;</span>
- <span
class="nt">&lt;version&gt;</span>1.8.1<span
class="nt">&lt;/version&gt;</span>
-<span class="nt">&lt;/dependency&gt;</span>
-<span class="nt">&lt;dependency&gt;</span>
- <span
class="nt">&lt;groupId&gt;</span>org.apache.flink<span
class="nt">&lt;/groupId&gt;</span>
- <span
class="nt">&lt;artifactId&gt;</span>flink-clients_2.11<span
class="nt">&lt;/artifactId&gt;</span>
- <span
class="nt">&lt;version&gt;</span>1.8.1<span
class="nt">&lt;/version&gt;</span>
-<span
class="nt">&lt;/dependency&gt;</span></code></pre></div>
-
-<p>You can find the binaries on the updated <a
href="/downloads.html">Downloads page</a>.</p>
-
-<p>List of resolved issues:</p>
-
-<h2> Sub-task
-</h2>
-<ul>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-10921">FLINK-10921</a>]
- Prioritize shard consumers in Kinesis Consumer by event time
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12617">FLINK-12617</a>]
- StandaloneJobClusterEntrypoint should default to random JobID for
non-HA setups
-</li>
-</ul>
-
-<h2> Bug
-</h2>
-<ul>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-9445">FLINK-9445</a>]
- scala-shell uses plain java command
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-10455">FLINK-10455</a>]
- Potential Kafka producer leak in case of failures
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-10941">FLINK-10941</a>]
- Slots prematurely released which still contain unconsumed data
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-11059">FLINK-11059</a>]
- JobMaster may continue using an invalid slot if releasing idle slot
meet a timeout
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-11107">FLINK-11107</a>]
- Avoid memory stateBackend to create arbitrary folders under HA path
when no checkpoint path configured
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-11897">FLINK-11897</a>]
- ExecutionGraphSuspendTest does not wait for all tasks to be submitted
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-11915">FLINK-11915</a>]
- DataInputViewStream skip returns wrong value
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-11987">FLINK-11987</a>]
- Kafka producer occasionally throws NullpointerException
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12009">FLINK-12009</a>]
- Wrong check message about heartbeat interval for HeartbeatServices
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12042">FLINK-12042</a>]
- RocksDBStateBackend mistakenly uses default filesystem
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12112">FLINK-12112</a>]
- AbstractTaskManagerProcessFailureRecoveryTest process output logging
does not work properly
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12132">FLINK-12132</a>]
- The example in /docs/ops/deployment/yarn_setup.md should be updated
due to the change FLINK-2021
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12184">FLINK-12184</a>]
- HistoryServerArchiveFetcher isn&#39;t compatible with old version
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12219">FLINK-12219</a>]
- Yarn application can&#39;t stop when flink job failed in per-job
yarn cluster mode
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12247">FLINK-12247</a>]
- fix NPE when writing an archive file to a FileSystem
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12260">FLINK-12260</a>]
- Slot allocation failure by taskmanager registration timeout and race
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12296">FLINK-12296</a>]
- Data loss silently in RocksDBStateBackend when more than one
operator(has states) chained in a single task
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12297">FLINK-12297</a>]
- Make ClosureCleaner recursive
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12301">FLINK-12301</a>]
- Scala value classes inside case classes cannot be serialized anymore
in Flink 1.8.0
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12342">FLINK-12342</a>]
- Yarn Resource Manager Acquires Too Many Containers
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12375">FLINK-12375</a>]
- flink-container job jar does not have read permissions
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12416">FLINK-12416</a>]
- Docker build script fails on symlink creation ln -s
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12544">FLINK-12544</a>]
- Deadlock while releasing memory and requesting segment concurrent in
SpillableSubpartition
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12547">FLINK-12547</a>]
- Deadlock when the task thread downloads jars using BlobClient
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12646">FLINK-12646</a>]
- Use reserved IP as unrouteable IP in RestClientTest
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12688">FLINK-12688</a>]
- Make serializer lazy initialization thread safe in StateDescriptor
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12740">FLINK-12740</a>]
- SpillableSubpartitionTest deadlocks on Travis
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12835">FLINK-12835</a>]
- Time conversion is wrong in ManualClock
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12863">FLINK-12863</a>]
- Race condition between slot offerings and AllocatedSlotReport
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12865">FLINK-12865</a>]
- State inconsistency between RM and TM on the slot status
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12871">FLINK-12871</a>]
- Wrong SSL setup examples in docs
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12895">FLINK-12895</a>]
-
TaskManagerProcessFailureBatchRecoveryITCase.testTaskManagerProcessFailure
failed on travis
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12896">FLINK-12896</a>]
- TaskCheckpointStatisticDetailsHandler uses wrong value for JobID
when archiving
-</li>
-</ul>
-
-<h2> Improvement
-</h2>
-<ul>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-11126">FLINK-11126</a>]
- Filter out AMRMToken in the TaskManager credentials
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12137">FLINK-12137</a>]
- Add more proper explanation on flink streaming connectors
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12169">FLINK-12169</a>]
- Improve Javadoc of MessageAcknowledgingSourceBase
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12378">FLINK-12378</a>]
- Consolidate FileSystem Documentation
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12391">FLINK-12391</a>]
- Add timeout to transfer.sh
-</li>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12539">FLINK-12539</a>]
- StreamingFileSink: Make the class extendable to customize for
different usecases
-</li>
-</ul>
-
-<h2> Test
-</h2>
-<ul>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12350">FLINK-12350</a>]
- RocksDBStateBackendTest doesn&#39;t cover the incremental
checkpoint code path
-</li>
-</ul>
-
-<h2> Task
-</h2>
-<ul>
-<li>[<a
href="https://issues.apache.org/jira/browse/FLINK-12460">FLINK-12460</a>]
- Change taskmanager.tmp.dirs to io.tmp.dirs in configuration docs
-</li>
-</ul>
-
-</description>
-<pubDate>Tue, 02 Jul 2019 14:00:00 +0200</pubDate>
-<link>https://flink.apache.org/news/2019/07/02/release-1.8.1.html</link>
-<guid isPermaLink="true">/news/2019/07/02/release-1.8.1.html</guid>
-</item>
-
</channel>
</rss>
diff --git a/content/blog/index.html b/content/blog/index.html
index c401e9893..58bb36c63 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a href="/2022/05/18/latency-part1.html">Getting
into Low-Latency Gears with Apache Flink - Part One</a></h2>
+
+ <p>18 May 2022
+ Jun Qin & Nico Kruber </p>
+
+ <p>This multi-part series of blog post presents a collection of
low-latency techniques in Flink. Part one starts with types of latency in Flink
and the way we measure the end-to-end latency, followed by a few techniques
that optimize latency directly.</p>
+
+ <p><a href="/2022/05/18/latency-part1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2022/05/11/release-table-store-0.1.0.html">Apache Flink Table Store
0.1.0 Release Announcement</a></h2>
@@ -360,22 +373,6 @@ exciting changes.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2022/01/31/release-statefun-3.2.0.html">Stateful Functions 3.2.0
Release Announcement</a></h2>
-
- <p>31 Jan 2022
- Till Rohrmann (<a href="https://twitter.com/stsffap";>@stsffap</a>)
& Igal Shilman (<a
href="https://twitter.com/IgalShilman";>@IgalShilman</a>)</p>
-
- <p><p>Stateful Functions is a cross-platform stack for building Stateful
Serverless applications, making it radically simpler to develop scalable,
consistent, and elastic distributed applications.
-This new release brings various improvements to the StateFun runtime, a leaner
way to specify StateFun module components, and a brand new JavaScript SDK!</p>
-
-</p>
-
- <p><a href="/news/2022/01/31/release-statefun-3.2.0.html">Continue
reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -408,6 +405,16 @@ This new release brings various improvements to the
StateFun runtime, a leaner w
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page10/index.html b/content/blog/page10/index.html
index 713b2fdfc..e08af5517 100644
--- a/content/blog/page10/index.html
+++ b/content/blog/page10/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2020/01/15/demo-fraud-detection.html">Advanced Flink Application
Patterns Vol.1: Case Study of a Fraud Detection System</a></h2>
+
+ <p>15 Jan 2020
+ Alexander Fedulov (<a
href="https://twitter.com/alex_fedulov";>@alex_fedulov</a>)</p>
+
+ <p>In this series of blog posts you will learn about three powerful
Flink patterns for building streaming applications.</p>
+
+ <p><a href="/news/2020/01/15/demo-fraud-detection.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2019/12/11/release-1.8.3.html">Apache Flink 1.8.3 Released</a></h2>
@@ -358,21 +371,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2019/07/02/release-1.8.1.html">Apache Flink 1.8.1 Released</a></h2>
-
- <p>02 Jul 2019
- Jincheng Sun (<a
href="https://twitter.com/sunjincheng121";>@sunjincheng121</a>)</p>
-
- <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.8 series.</p>
-
-</p>
-
- <p><a href="/news/2019/07/02/release-1.8.1.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -405,6 +403,16 @@
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page11/index.html b/content/blog/page11/index.html
index 2784e86ce..34765b5e1 100644
--- a/content/blog/page11/index.html
+++ b/content/blog/page11/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2019/07/02/release-1.8.1.html">Apache Flink 1.8.1 Released</a></h2>
+
+ <p>02 Jul 2019
+ Jincheng Sun (<a
href="https://twitter.com/sunjincheng121";>@sunjincheng121</a>)</p>
+
+ <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.8 series.</p>
+
+</p>
+
+ <p><a href="/news/2019/07/02/release-1.8.1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a href="/2019/06/26/broadcast-state.html">A
Practical Guide to Broadcast State in Apache Flink</a></h2>
@@ -357,19 +372,6 @@ for more details.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2019/02/25/monitoring-best-practices.html">Monitoring Apache Flink
Applications 101</a></h2>
-
- <p>25 Feb 2019
- Konstantin Knauf (<a
href="https://twitter.com/snntrable";>@snntrable</a>)</p>
-
- <p>The monitoring of business-critical applications is a crucial aspect
of a production deployment. It ensures that any degradation or downtime is
immediately identified and can be resolved as quickly as possible. In this
post, we discuss the most important metrics that indicate healthy Flink
applications.</p>
-
- <p><a href="/news/2019/02/25/monitoring-best-practices.html">Continue
reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -402,6 +404,16 @@ for more details.</p>
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page12/index.html b/content/blog/page12/index.html
index 9d391d443..b0760e7fd 100644
--- a/content/blog/page12/index.html
+++ b/content/blog/page12/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2019/02/25/monitoring-best-practices.html">Monitoring Apache Flink
Applications 101</a></h2>
+
+ <p>25 Feb 2019
+ Konstantin Knauf (<a
href="https://twitter.com/snntrable";>@snntrable</a>)</p>
+
+ <p>The monitoring of business-critical applications is a crucial aspect
of a production deployment. It ensures that any degradation or downtime is
immediately identified and can be resolved as quickly as possible. In this
post, we discuss the most important metrics that indicate healthy Flink
applications.</p>
+
+ <p><a href="/news/2019/02/25/monitoring-best-practices.html">Continue
reading »</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2019/02/25/release-1.6.4.html">Apache Flink 1.6.4 Released</a></h2>
@@ -367,21 +380,6 @@ Please check the <a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2018/09/20/release-1.6.1.html">Apache Flink 1.6.1 Released</a></h2>
-
- <p>20 Sep 2018
- </p>
-
- <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.6 series.</p>
-
-</p>
-
- <p><a href="/news/2018/09/20/release-1.6.1.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -414,6 +412,16 @@ Please check the <a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page13/index.html b/content/blog/page13/index.html
index a12ffeb8f..8c80f12a8 100644
--- a/content/blog/page13/index.html
+++ b/content/blog/page13/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2018/09/20/release-1.6.1.html">Apache Flink 1.6.1 Released</a></h2>
+
+ <p>20 Sep 2018
+ </p>
+
+ <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.6 series.</p>
+
+</p>
+
+ <p><a href="/news/2018/09/20/release-1.6.1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2018/09/20/release-1.5.4.html">Apache Flink 1.5.4 Released</a></h2>
@@ -365,21 +380,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2018/02/15/release-1.4.1.html">Apache Flink 1.4.1 Released</a></h2>
-
- <p>15 Feb 2018
- </p>
-
- <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.4 series.</p>
-
-</p>
-
- <p><a href="/news/2018/02/15/release-1.4.1.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -412,6 +412,16 @@
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page14/index.html b/content/blog/page14/index.html
index 8b78d6e2e..d882f1a53 100644
--- a/content/blog/page14/index.html
+++ b/content/blog/page14/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2018/02/15/release-1.4.1.html">Apache Flink 1.4.1 Released</a></h2>
+
+ <p>15 Feb 2018
+ </p>
+
+ <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.4 series.</p>
+
+</p>
+
+ <p><a href="/news/2018/02/15/release-1.4.1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/features/2018/01/30/incremental-checkpointing.html">Managing Large State
in Apache Flink: An Intro to Incremental Checkpointing</a></h2>
@@ -366,21 +381,6 @@ what’s coming in Flink 1.4.0 as well as a preview of what
the Flink community
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2017/04/26/release-1.2.1.html">Apache Flink 1.2.1 Released</a></h2>
-
- <p>26 Apr 2017
- </p>
-
- <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.2 series.</p>
-
-</p>
-
- <p><a href="/news/2017/04/26/release-1.2.1.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -413,6 +413,16 @@ what’s coming in Flink 1.4.0 as well as a preview of what
the Flink community
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page15/index.html b/content/blog/page15/index.html
index ad50ac63d..c145a330d 100644
--- a/content/blog/page15/index.html
+++ b/content/blog/page15/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2017/04/26/release-1.2.1.html">Apache Flink 1.2.1 Released</a></h2>
+
+ <p>26 Apr 2017
+ </p>
+
+ <p><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.2 series.</p>
+
+</p>
+
+ <p><a href="/news/2017/04/26/release-1.2.1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2017/04/04/dynamic-tables.html">Continuous Queries on Dynamic
Tables</a></h2>
@@ -360,21 +375,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2016/08/11/release-1.1.1.html">Flink 1.1.1 Released</a></h2>
-
- <p>11 Aug 2016
- </p>
-
- <p><p>Today, the Flink community released Flink version 1.1.1.</p>
-
-</p>
-
- <p><a href="/news/2016/08/11/release-1.1.1.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -407,6 +407,16 @@
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page16/index.html b/content/blog/page16/index.html
index 7a46d6c6c..5e0cbe88b 100644
--- a/content/blog/page16/index.html
+++ b/content/blog/page16/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2016/08/11/release-1.1.1.html">Flink 1.1.1 Released</a></h2>
+
+ <p>11 Aug 2016
+ </p>
+
+ <p><p>Today, the Flink community released Flink version 1.1.1.</p>
+
+</p>
+
+ <p><a href="/news/2016/08/11/release-1.1.1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2016/08/08/release-1.1.0.html">Announcing Apache Flink
1.1.0</a></h2>
@@ -364,19 +379,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2015/12/18/a-year-in-review.html">Flink 2015: A year in review, and
a lookout to 2016</a></h2>
-
- <p>18 Dec 2015 by Robert Metzger (<a
href="https://twitter.com/";>@rmetzger_</a>)
- </p>
-
- <p><p>With 2015 ending, we thought that this would be good time to
reflect on the amazing work done by the Flink community over this past year,
and how much this community has grown.</p></p>
-
- <p><a href="/news/2015/12/18/a-year-in-review.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -409,6 +411,16 @@
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page17/index.html b/content/blog/page17/index.html
index a4556c250..f7a3b86d7 100644
--- a/content/blog/page17/index.html
+++ b/content/blog/page17/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2015/12/18/a-year-in-review.html">Flink 2015: A year in review, and
a lookout to 2016</a></h2>
+
+ <p>18 Dec 2015 by Robert Metzger (<a
href="https://twitter.com/";>@rmetzger_</a>)
+ </p>
+
+ <p><p>With 2015 ending, we thought that this would be good time to
reflect on the amazing work done by the Flink community over this past year,
and how much this community has grown.</p></p>
+
+ <p><a href="/news/2015/12/18/a-year-in-review.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2015/12/11/storm-compatibility.html">Storm Compatibility in Apache
Flink: How to run existing Storm topologies on Flink</a></h2>
@@ -370,19 +383,6 @@ vertex-centric or gather-sum-apply to Flink dataflows.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2015/05/14/Community-update-April.html">April 2015 in the Flink
community</a></h2>
-
- <p>14 May 2015 by Kostas Tzoumas (<a
href="https://twitter.com/";>@kostas_tzoumas</a>)
- </p>
-
- <p><p>The monthly update from the Flink community. Including the
availability of a new preview release, lots of meetups and conference talks and
a great interview about Flink.</p></p>
-
- <p><a href="/news/2015/05/14/Community-update-April.html">Continue
reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -415,6 +415,16 @@ vertex-centric or gather-sum-apply to Flink dataflows.</p>
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page18/index.html b/content/blog/page18/index.html
index 3e2f6fd8d..96bfb087a 100644
--- a/content/blog/page18/index.html
+++ b/content/blog/page18/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2015/05/14/Community-update-April.html">April 2015 in the Flink
community</a></h2>
+
+ <p>14 May 2015 by Kostas Tzoumas (<a
href="https://twitter.com/";>@kostas_tzoumas</a>)
+ </p>
+
+ <p><p>The monthly update from the Flink community. Including the
availability of a new preview release, lots of meetups and conference talks and
a great interview about Flink.</p></p>
+
+ <p><a href="/news/2015/05/14/Community-update-April.html">Continue
reading »</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2015/05/11/Juggling-with-Bits-and-Bytes.html">Juggling with Bits
and Bytes</a></h2>
@@ -376,21 +389,6 @@ and offers a new API including definition of flexible
windows.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2014/11/18/hadoop-compatibility.html">Hadoop Compatibility in
Flink</a></h2>
-
- <p>18 Nov 2014 by Fabian Hüske (<a
href="https://twitter.com/";>@fhueske</a>)
- </p>
-
- <p><p><a href="http://hadoop.apache.org";>Apache Hadoop</a> is an
industry standard for scalable analytical data processing. Many data analysis
applications have been implemented as Hadoop MapReduce jobs and run in clusters
around the world. Apache Flink can be an alternative to MapReduce and improves
it in many dimensions. Among other features, Flink provides much better
performance and offers APIs in Java and Scala, which are very easy to use.
Similar to Hadoop, Flink’s APIs provi [...]
-
-</p>
-
- <p><a href="/news/2014/11/18/hadoop-compatibility.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -423,6 +421,16 @@ and offers a new API including definition of flexible
windows.</p>
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page19/index.html b/content/blog/page19/index.html
index 0a6f55882..ae99d7d21 100644
--- a/content/blog/page19/index.html
+++ b/content/blog/page19/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2014/11/18/hadoop-compatibility.html">Hadoop Compatibility in
Flink</a></h2>
+
+ <p>18 Nov 2014 by Fabian Hüske (<a
href="https://twitter.com/";>@fhueske</a>)
+ </p>
+
+ <p><p><a href="http://hadoop.apache.org";>Apache Hadoop</a> is an
industry standard for scalable analytical data processing. Many data analysis
applications have been implemented as Hadoop MapReduce jobs and run in clusters
around the world. Apache Flink can be an alternative to MapReduce and improves
it in many dimensions. Among other features, Flink provides much better
performance and offers APIs in Java and Scala, which are very easy to use.
Similar to Hadoop, Flink’s APIs provi [...]
+
+</p>
+
+ <p><a href="/news/2014/11/18/hadoop-compatibility.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2014/11/04/release-0.7.0.html">Apache Flink 0.7.0 available</a></h2>
@@ -327,6 +342,16 @@ academic and open source project that Flink originates
from.</p>
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html
index 7b6154957..3a800502b 100644
--- a/content/blog/page2/index.html
+++ b/content/blog/page2/index.html
@@ -232,6 +232,22 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2022/01/31/release-statefun-3.2.0.html">Stateful Functions 3.2.0
Release Announcement</a></h2>
+
+ <p>31 Jan 2022
+ Till Rohrmann (<a href="https://twitter.com/stsffap";>@stsffap</a>)
& Igal Shilman (<a
href="https://twitter.com/IgalShilman";>@IgalShilman</a>)</p>
+
+ <p><p>Stateful Functions is a cross-platform stack for building Stateful
Serverless applications, making it radically simpler to develop scalable,
consistent, and elastic distributed applications.
+This new release brings various improvements to the StateFun runtime, a leaner
way to specify StateFun module components, and a brand new JavaScript SDK!</p>
+
+</p>
+
+ <p><a href="/news/2022/01/31/release-statefun-3.2.0.html">Continue
reading »</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/2022/01/20/pravega-connector-101.html">Pravega Flink Connector
101</a></h2>
@@ -353,19 +369,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/2021/10/26/sort-shuffle-part2.html">Sort-Based Blocking Shuffle
Implementation in Flink - Part Two</a></h2>
-
- <p>26 Oct 2021
- Yingjie Cao (Kevin) & Daisy Tsang </p>
-
- <p>Flink has implemented the sort-based blocking shuffle (FLIP-148) for
batch data processing. In this blog post, we will take a close look at the
design & implementation details and see what we can gain from it.</p>
-
- <p><a href="/2021/10/26/sort-shuffle-part2.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -398,6 +401,16 @@
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html
index 0f8859f69..c3e30f676 100644
--- a/content/blog/page3/index.html
+++ b/content/blog/page3/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/2021/10/26/sort-shuffle-part2.html">Sort-Based Blocking Shuffle
Implementation in Flink - Part Two</a></h2>
+
+ <p>26 Oct 2021
+ Yingjie Cao (Kevin) & Daisy Tsang </p>
+
+ <p>Flink has implemented the sort-based blocking shuffle (FLIP-148) for
batch data processing. In this blog post, we will take a close look at the
design & implementation details and see what we can gain from it.</p>
+
+ <p><a href="/2021/10/26/sort-shuffle-part2.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/2021/10/26/sort-shuffle-part1.html">Sort-Based Blocking Shuffle
Implementation in Flink - Part One</a></h2>
@@ -371,21 +384,6 @@ This new release brings various improvements to the
StateFun runtime, a leaner w
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2021/08/06/release-1.12.5.html">Apache Flink 1.12.5
Released</a></h2>
-
- <p>06 Aug 2021
- Jingsong Lee </p>
-
- <p><p>The Apache Flink community released the next bugfix version of the
Apache Flink 1.12 series.</p>
-
-</p>
-
- <p><a href="/news/2021/08/06/release-1.12.5.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -418,6 +416,16 @@ This new release brings various improvements to the
StateFun runtime, a leaner w
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index d5a9d03e8..dc0e075ae 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2021/08/06/release-1.12.5.html">Apache Flink 1.12.5
Released</a></h2>
+
+ <p>06 Aug 2021
+ Jingsong Lee </p>
+
+ <p><p>The Apache Flink community released the next bugfix version of the
Apache Flink 1.12 series.</p>
+
+</p>
+
+ <p><a href="/news/2021/08/06/release-1.12.5.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a href="/2021/07/07/backpressure.html">How to
identify the source of backpressure?</a></h2>
@@ -361,19 +376,6 @@ to develop scalable, consistent, and elastic distributed
applications.</p>
<hr>
- <article>
- <h2 class="blog-title"><a href="/2021/02/10/native-k8s-with-ha.html">How
to natively deploy Flink on Kubernetes with High-Availability (HA)</a></h2>
-
- <p>10 Feb 2021
- Yang Wang </p>
-
- <p>Kubernetes provides built-in functionalities that Flink can leverage
for JobManager failover. In Flink 1.12 (FLIP-144), the community implemented a
Kubernetes High Availability (HA) service as an alternative to ZooKeeper for
highly available production setups. In this blogpost, we will have a close look
at how to deploy Flink applications natively on Kubernetes cluster with HA.</p>
-
- <p><a href="/2021/02/10/native-k8s-with-ha.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -406,6 +408,16 @@ to develop scalable, consistent, and elastic distributed
applications.</p>
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html
index c462d8388..2aa81c90e 100644
--- a/content/blog/page5/index.html
+++ b/content/blog/page5/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a href="/2021/02/10/native-k8s-with-ha.html">How
to natively deploy Flink on Kubernetes with High-Availability (HA)</a></h2>
+
+ <p>10 Feb 2021
+ Yang Wang </p>
+
+ <p>Kubernetes provides built-in functionalities that Flink can leverage
for JobManager failover. In Flink 1.12 (FLIP-144), the community implemented a
Kubernetes High Availability (HA) service as an alternative to ZooKeeper for
highly available production setups. In this blogpost, we will have a close look
at how to deploy Flink applications natively on Kubernetes cluster with HA.</p>
+
+ <p><a href="/2021/02/10/native-k8s-with-ha.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2021/01/29/release-1.10.3.html">Apache Flink 1.10.3
Released</a></h2>
@@ -357,21 +370,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2020/11/11/release-statefun-2.2.1.html">Stateful Functions 2.2.1
Release Announcement</a></h2>
-
- <p>11 Nov 2020
- Tzu-Li (Gordon) Tai (<a
href="https://twitter.com/tzulitai";>@tzulitai</a>)</p>
-
- <p><p>The Apache Flink community released the first bugfix release of
the Stateful Functions (StateFun) 2.2 series, version 2.2.1.</p>
-
-</p>
-
- <p><a href="/news/2020/11/11/release-statefun-2.2.1.html">Continue
reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -404,6 +402,16 @@
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page6/index.html b/content/blog/page6/index.html
index cd9f71bec..a45350467 100644
--- a/content/blog/page6/index.html
+++ b/content/blog/page6/index.html
@@ -232,6 +232,21 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2020/11/11/release-statefun-2.2.1.html">Stateful Functions 2.2.1
Release Announcement</a></h2>
+
+ <p>11 Nov 2020
+ Tzu-Li (Gordon) Tai (<a
href="https://twitter.com/tzulitai";>@tzulitai</a>)</p>
+
+ <p><p>The Apache Flink community released the first bugfix release of
the Stateful Functions (StateFun) 2.2 series, version 2.2.1.</p>
+
+</p>
+
+ <p><a href="/news/2020/11/11/release-statefun-2.2.1.html">Continue
reading »</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/2020/10/15/from-aligned-to-unaligned-checkpoints-part-1.html">From
Aligned to Unaligned Checkpoints - Part 1: Checkpoints, Alignment, and
Backpressure</a></h2>
@@ -361,19 +376,6 @@ as well as increased observability for operational
purposes.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2020/08/06/external-resource.html">Accelerating your workload with
GPU and other external resources</a></h2>
-
- <p>06 Aug 2020
- Yangze Guo </p>
-
- <p>This post introduces the new External Resource Framework in Flink
1.11 and take GPU as an example to show how to accelerate your workload with
external resources.</p>
-
- <p><a href="/news/2020/08/06/external-resource.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -406,6 +408,16 @@ as well as increased observability for operational
purposes.</p>
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page7/index.html b/content/blog/page7/index.html
index c0530af0c..8fc433313 100644
--- a/content/blog/page7/index.html
+++ b/content/blog/page7/index.html
@@ -232,6 +232,19 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2020/08/06/external-resource.html">Accelerating your workload with
GPU and other external resources</a></h2>
+
+ <p>06 Aug 2020
+ Yangze Guo </p>
+
+ <p>This post introduces the new External Resource Framework in Flink
1.11 and take GPU as an example to show how to accelerate your workload with
external resources.</p>
+
+ <p><a href="/news/2020/08/06/external-resource.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/2020/08/04/pyflink-pandas-udf-support-flink.html">PyFlink: The
integration of Pandas into PyFlink</a></h2>
@@ -363,23 +376,6 @@ perform streaming data visualization via Flink on Zeppelin
and how to use Apache
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2020/06/15/flink-on-zeppelin-part1.html">Flink on Zeppelin
Notebooks for Interactive Data Analysis - Part 1</a></h2>
-
- <p>15 Jun 2020
- Jeff Zhang (<a href="https://twitter.com/zjffdu";>@zjffdu</a>)</p>
-
- <p><p>The latest release of <a
href="https://zeppelin.apache.org/";>Apache Zeppelin</a> comes with a redesigned
interpreter for Apache Flink (version Flink 1.10+ is only supported moving
forward)
-that allows developers to use Flink directly on Zeppelin notebooks for
interactive data analysis. I wrote 2 posts about how to use Flink in Zeppelin.
This is part-1 where I explain how the Flink interpreter in Zeppelin works,
-and provide a tutorial for running Streaming ETL with Flink on Zeppelin.</p>
-
-</p>
-
- <p><a href="/news/2020/06/15/flink-on-zeppelin-part1.html">Continue
reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -412,6 +408,16 @@ and provide a tutorial for running Streaming ETL with
Flink on Zeppelin.</p>
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page8/index.html b/content/blog/page8/index.html
index 5637edd10..8cfec02fb 100644
--- a/content/blog/page8/index.html
+++ b/content/blog/page8/index.html
@@ -232,6 +232,23 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2020/06/15/flink-on-zeppelin-part1.html">Flink on Zeppelin
Notebooks for Interactive Data Analysis - Part 1</a></h2>
+
+ <p>15 Jun 2020
+ Jeff Zhang (<a href="https://twitter.com/zjffdu";>@zjffdu</a>)</p>
+
+ <p><p>The latest release of <a
href="https://zeppelin.apache.org/";>Apache Zeppelin</a> comes with a redesigned
interpreter for Apache Flink (version Flink 1.10+ is only supported moving
forward)
+that allows developers to use Flink directly on Zeppelin notebooks for
interactive data analysis. I wrote 2 posts about how to use Flink in Zeppelin.
This is part-1 where I explain how the Flink interpreter in Zeppelin works,
+and provide a tutorial for running Streaming ETL with Flink on Zeppelin.</p>
+
+</p>
+
+ <p><a href="/news/2020/06/15/flink-on-zeppelin-part1.html">Continue
reading »</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2020/06/11/community-update.html">Flink Community Update -
June'20</a></h2>
@@ -355,22 +372,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2020/04/07/release-statefun-2.0.0.html">Stateful Functions 2.0 - An
Event-driven Database on Apache Flink</a></h2>
-
- <p>07 Apr 2020
- Stephan Ewen (<a
href="https://twitter.com/stephanewen";>@stephanewen</a>)</p>
-
- <p><p>Today, we are announcing the release of Stateful Functions
(StateFun) 2.0 — the first release of Stateful Functions as part of the Apache
Flink project.
-This release marks a big milestone: Stateful Functions 2.0 is not only an API
update, but the <strong>first version of an event-driven database</strong> that
is built on Apache Flink.</p>
-
-</p>
-
- <p><a href="/news/2020/04/07/release-statefun-2.0.0.html">Continue
reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -403,6 +404,16 @@ This release marks a big milestone: Stateful Functions 2.0
is not only an API up
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/blog/page9/index.html b/content/blog/page9/index.html
index 82d4a2847..2c9996f22 100644
--- a/content/blog/page9/index.html
+++ b/content/blog/page9/index.html
@@ -232,6 +232,22 @@
<div class="col-sm-8">
<!-- Blog posts -->
+ <article>
+ <h2 class="blog-title"><a
href="/news/2020/04/07/release-statefun-2.0.0.html">Stateful Functions 2.0 - An
Event-driven Database on Apache Flink</a></h2>
+
+ <p>07 Apr 2020
+ Stephan Ewen (<a
href="https://twitter.com/stephanewen";>@stephanewen</a>)</p>
+
+ <p><p>Today, we are announcing the release of Stateful Functions
(StateFun) 2.0 — the first release of Stateful Functions as part of the Apache
Flink project.
+This release marks a big milestone: Stateful Functions 2.0 is not only an API
update, but the <strong>first version of an event-driven database</strong> that
is built on Apache Flink.</p>
+
+</p>
+
+ <p><a href="/news/2020/04/07/release-statefun-2.0.0.html">Continue
reading »</a></p>
+ </article>
+
+ <hr>
+
<article>
<h2 class="blog-title"><a
href="/news/2020/04/01/community-update.html">Flink Community Update -
April'20</a></h2>
@@ -355,19 +371,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2020/01/15/demo-fraud-detection.html">Advanced Flink Application
Patterns Vol.1: Case Study of a Fraud Detection System</a></h2>
-
- <p>15 Jan 2020
- Alexander Fedulov (<a
href="https://twitter.com/alex_fedulov";>@alex_fedulov</a>)</p>
-
- <p>In this series of blog posts you will learn about three powerful
Flink patterns for building streaming applications.</p>
-
- <p><a href="/news/2020/01/15/demo-fraud-detection.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -400,6 +403,16 @@
<ul id="markdown-toc">
+ <li><a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></li>
diff --git a/content/index.html b/content/index.html
index 742e062dd..69da5427f 100644
--- a/content/index.html
+++ b/content/index.html
@@ -397,6 +397,9 @@
<dl>
+ <dt> <a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></dt>
+ <dd>This multi-part series of blog post presents a collection of
low-latency techniques in Flink. Part one starts with types of latency in Flink
and the way we measure the end-to-end latency, followed by a few techniques
that optimize latency directly.</dd>
+
<dt> <a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></dt>
<dd><p>The Apache Flink community is pleased to announce the preview
release of the
<a href="https://github.com/apache/flink-table-store";>Apache Flink Table
Store</a> (0.1.0).</p>
@@ -411,15 +414,6 @@
<dt> <a href="/2022/05/06/restore-modes.html">Improvements to Flink
operations: Snapshots Ownership and Savepoint Formats</a></dt>
<dd>This post will outline the journey of improving snapshotting in
past releases and the upcoming improvements in Flink 1.15, which includes
making it possible to take savepoints in the native state backend specific
format as well as clarifying snapshots ownership.</dd>
-
- <dt> <a href="/news/2022/05/05/1.15-announcement.html">Announcing the
Release of Apache Flink 1.15</a></dt>
- <dd><p>Thanks to our well-organized and open community, Apache Flink
continues
-<a href="https://www.apache.org/foundation/docs/FY2021AnnualReport.pdf";>to
grow</a> as a
-technology and remain one of the most active projects in
-the Apache community. With the release of Flink 1.15, we are proud to announce
a number of
-exciting changes.</p>
-
-</dd>
</dl>
diff --git a/content/zh/index.html b/content/zh/index.html
index 7702729ae..e6d695f5e 100644
--- a/content/zh/index.html
+++ b/content/zh/index.html
@@ -394,6 +394,9 @@
<dl>
+ <dt> <a href="/2022/05/18/latency-part1.html">Getting into Low-Latency
Gears with Apache Flink - Part One</a></dt>
+ <dd>This multi-part series of blog post presents a collection of
low-latency techniques in Flink. Part one starts with types of latency in Flink
and the way we measure the end-to-end latency, followed by a few techniques
that optimize latency directly.</dd>
+
<dt> <a href="/news/2022/05/11/release-table-store-0.1.0.html">Apache
Flink Table Store 0.1.0 Release Announcement</a></dt>
<dd><p>The Apache Flink community is pleased to announce the preview
release of the
<a href="https://github.com/apache/flink-table-store";>Apache Flink Table
Store</a> (0.1.0).</p>
@@ -408,15 +411,6 @@
<dt> <a href="/2022/05/06/restore-modes.html">Improvements to Flink
operations: Snapshots Ownership and Savepoint Formats</a></dt>
<dd>This post will outline the journey of improving snapshotting in
past releases and the upcoming improvements in Flink 1.15, which includes
making it possible to take savepoints in the native state backend specific
format as well as clarifying snapshots ownership.</dd>
-
- <dt> <a href="/news/2022/05/05/1.15-announcement.html">Announcing the
Release of Apache Flink 1.15</a></dt>
- <dd><p>Thanks to our well-organized and open community, Apache Flink
continues
-<a href="https://www.apache.org/foundation/docs/FY2021AnnualReport.pdf";>to
grow</a> as a
-technology and remain one of the most active projects in
-the Apache community. With the release of Flink 1.15, we are proud to announce
a number of
-exciting changes.</p>
-
-</dd>
</dl>
