This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit f75d2fdc724312fdb701f07cdb96c5a2346f0884
Author: Mergebot <merge...@apache.org>
AuthorDate: Mon Feb 19 05:30:02 2018 -0800

    Prepare repository for deployment.
---
 content/blog/2018/02/19/beam-2.3.0.html | 237 ++++++++++++++++++
 content/blog/index.html                 |  17 ++
 content/feed.xml                        | 410 ++++++++------------------------
 content/index.html                      |  10 +-
 4 files changed, 356 insertions(+), 318 deletions(-)

diff --git a/content/blog/2018/02/19/beam-2.3.0.html 
b/content/blog/2018/02/19/beam-2.3.0.html
new file mode 100644
index 0000000..af8c605
--- /dev/null
+++ b/content/blog/2018/02/19/beam-2.3.0.html
@@ -0,0 +1,237 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+  <meta charset="utf-8">
+  <meta http-equiv="X-UA-Compatible" content="IE=edge">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>Apache Beam 2.3.0</title>
+  <meta name="description" content="We are glad to present the new 2.3.0 
release of Beam. This release includesmultiple fixes and new functionalities.">
+  <link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400"; 
rel="stylesheet">
+  <link rel="stylesheet" href="/css/site.css">
+  <script 
src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js";></script>
+  <script src="/js/bootstrap.min.js"></script>
+  <script src="/js/language-switch.js"></script>
+  <script src="/js/fix-menu.js"></script>
+  <script src="/js/section-nav.js"></script>
+  <script src="/js/page-nav.js"></script>
+  <link rel="canonical" 
href="https://beam.apache.org/blog/2018/02/19/beam-2.3.0.html"; 
data-proofer-ignore>
+  <link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
+  <link rel="alternate" type="application/rss+xml" title="Apache Beam" 
href="https://beam.apache.org/feed.xml";>
+  <script>
+    
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new 
Date();a=s.createElement(o),
+    
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+    
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+    ga('create', 'UA-73650088-1', 'auto');
+    ga('send', 'pageview');
+  </script>
+</head>
+
+  <body class="body ">
+    <nav class="header navbar navbar-fixed-top">
+    <div class="navbar-header">
+      <button type="button" class="navbar-toggle" aria-expanded="false" 
aria-controls="navbar">
+        <span class="sr-only">Toggle navigation</span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+      </button>
+
+      <a href="/" class="navbar-brand" >
+        <img alt="Brand" style="height: 25px" 
src="/images/beam_logo_navbar.png">
+      </a>
+    </div>
+
+    <div class="navbar-mask closed"></div>
+
+    <div id="navbar" class="navbar-container closed">
+      <ul class="nav navbar-nav">
+        <li>
+          <a href="/get-started/beam-overview/">Get Started</a>
+        </li>
+        <li>
+          <a href="/documentation/">Documentation</a>
+        </li>
+        <li>
+          <a href="/documentation/sdks/java/">SDKS</a>
+        </li>
+        <li>
+          <a href="/documentation/runners/capability-matrix/">RUNNERS</a>
+        </li>
+        <li>
+          <a href="/contribute/">Contribute</a>
+        </li>
+        <li><a href="/blog">Blog</a></li>
+      </ul>
+      <ul class="nav navbar-nav navbar-right">
+        <li class="dropdown">
+          <a href="#" class="dropdown-toggle" data-toggle="dropdown" 
role="button" aria-haspopup="true" aria-expanded="false"><img 
src="https://www.apache.org/foundation/press/kit/feather_small.png"; alt="Apache 
Logo" style="height:20px;"><span class="caret"></span></a>
+          <ul class="dropdown-menu dropdown-menu-right">
+            <li><a href="http://www.apache.org/";>ASF Homepage</a></li>
+            <li><a href="http://www.apache.org/licenses/";>License</a></li>
+            <li><a href="http://www.apache.org/security/";>Security</a></li>
+            <li><a 
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+            <li><a 
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+            <li><a 
href="https://www.apache.org/foundation/policies/conduct";>Code of 
Conduct</a></li>
+          </ul>
+        </li>
+      </ul>
+    </div>
+</nav>
+
+    <div class="body__contained">
+      
+
+<article class="post" itemscope itemtype="http://schema.org/BlogPosting";>
+
+  <header class="post-header">
+    <h1 class="post-title" itemprop="name headline">Apache Beam 2.3.0</h1>
+    <p class="post-meta"><time datetime="2018-02-19T00:00:01-08:00" 
itemprop="datePublished">Feb 19, 2018</time> •  Ismaël Mejía [<a 
href="https://twitter.com/iemejia";>@iemejia</a>]
+</p>
+  </header>
+
+  <div class="post-content" itemprop="articleBody">
+    <p>We are glad to present the new 2.3.0 release of Beam. This release 
includes
+multiple fixes and new functionalities. <!--more--> For more information
+please check the detailed release notes.</p>
+
+<h1 id="new-features--improvements">New Features / Improvements</h1>
+
+<h2 id="beam-moves-to-java-8">Beam moves to Java 8</h2>
+
+<p>The supported version of Java for Beam is now Java 8. The code and examples 
have
+been refactored to use multiple of the advantages of the language, e.g. 
lambdas,
+streams, improved type inference, etc.</p>
+
+<h2 id="spark-runner-is-now-based-on-spark-2x">Spark runner is now based on 
Spark 2.x</h2>
+
+<p>Spark runner moves forward into the Spark 2.x development line, this would 
allow
+to benefit of improved performance, as well as open the runner for future
+compatibility with the Structured Streaming APIs. Notice that support for Spark
+1.x is finished with this release.</p>
+
+<h2 id="amazon-web-services-s3-filesystem-support">Amazon Web Services S3 
Filesystem support</h2>
+
+<p>Beam already supported AWS S3 via HadoopFileSystem, but this version brings 
a
+native implementation with the corresponding performance advantages of the S3
+filesystem.</p>
+
+<h2 id="general-purpose-writing-to-files">General-purpose writing to files</h2>
+
+<p>This release contains a new transform, FileIO.write() / writeDynamic() that
+implements a general-purpose fluent and Java8-friendly API for writing to files
+using a FileIO.Sink. This API has similar capabilities to DynamicDestinations
+APIs from Beam 2.2 but is much easier to use and extend. The 
DynamicDestinations
+APIs for writing to files are deprecated by it, as is FileBasedSink.</p>
+
+<h2 id="splittable-dofn-support-on-the-python-sdk">Splittable DoFn support on 
the Python SDK</h2>
+
+<p>This release adds the Splittable DoFn API for Python SDK and adds Splittable
+DoFn support for Python streaming DirectRunner.</p>
+
+<h2 id="portability">Portability</h2>
+
+<p>Progress continues to being able to execute Python on runners other then 
Google
+Cloud Dataflow and the Go SDK on any runner.</p>
+
+<h1 id="miscellaneous-fixes">Miscellaneous Fixes</h1>
+
+<h2 id="sdks">SDKs</h2>
+
+<ul>
+  <li>MapElements and FlatMapElements support using side inputs using the new
+interface Contextful.Fn. For library authors, this interface is the
+recommended choice for user-code callbacks that may use side inputs.</li>
+  <li>Introduces the family of Reify transforms for converting between 
explicit and
+implicit representations of various Beam entities.</li>
+  <li>Introduces two transforms for approximate sketching of data: Count-Min 
Sketch
+(approximate element frequency estimation) and HyperLogLog (approximate
+cardinality estimation).</li>
+</ul>
+
+<h2 id="runners">Runners</h2>
+
+<ul>
+  <li>Staging files on Dataflow shows progress</li>
+  <li>Flink runner is based now on Flink version 1.4.0</li>
+</ul>
+
+<h2 id="ios">IOs</h2>
+
+<ul>
+  <li>BigtableIO now supports ValueProvider configuration</li>
+  <li>BigQueryIO supports writing bounded collections to tables with partition
+decorators</li>
+  <li>KafkaIO moves to version 1.0 (it is still backwards compatible with 
versions &gt;= 0.9.x.x)</li>
+  <li>Added IO source for VCF files (Python)</li>
+  <li>Added support for backoff on deadlocks in JdbcIO.write() and connection
+improvement</li>
+  <li>Improved performance of KinesisIO.read()</li>
+  <li>Many improvements to TikaIO</li>
+</ul>
+
+<h1 id="list-of-contributors">List of Contributors</h1>
+
+<p>According to git shortlog, the following 80 people contributed to the 2.3.0 
release. Thank you to all contributors!</p>
+
+<p>Ahmet Altay, Alan Myrvold, Alex Amato, Alexey Romanenko, Ankur Goenka, 
Anton Kedin, Arnaud Fournier, Alexey Romanenko, Asha Rostamianfar, Ben 
Chambers, Ben Sidhom, Bill Neubauer, Brian Foo, cclauss, Chamikara Jayalath, 
Charles Chen, Colm O hEigeartaigh, Daniel Oliveira, Dariusz Aniszewski, David 
Cavazos, David Sabater, David Sabater Dinter, Dawid Wysakowicz, Dmytro Ivanov, 
Etienne Chauchot, Eugene Kirpichov, Exprosed, Grzegorz Kołakowski, Henning 
Rohde, Holden Karau, Huygaa Batsaikhan [...]
+
+
+  </div>
+
+</article>
+
+    </div>
+    <footer class="footer">
+  <div class="footer__contained">
+    <div class="footer__cols">
+      <div class="footer__cols__col">
+        <div class="footer__cols__col__logo">
+          <img src="/images/beam_logo_circle.svg" class="footer__logo" 
alt="Beam logo">
+        </div>
+        <div class="footer__cols__col__logo">
+          <img src="/images/apache_logo_circle.svg" class="footer__logo" 
alt="Apache logo">
+        </div>
+      </div>
+      <div class="footer__cols__col footer__cols__col--md">
+        <div class="footer__cols__col__title">Start</div>
+        <div class="footer__cols__col__link"><a 
href="/get-started/beam-overview/">Overview</a></div>
+        <div class="footer__cols__col__link"><a 
href="/get-started/quickstart-java/">Quickstart (Java)</a></div>
+        <div class="footer__cols__col__link"><a 
href="/get-started/quickstart-py/">Quickstart (Python)</a></div>
+        <div class="footer__cols__col__link"><a 
href="/get-started/downloads/">Downloads</a></div>
+      </div>
+      <div class="footer__cols__col footer__cols__col--md">
+        <div class="footer__cols__col__title">Docs</div>
+        <div class="footer__cols__col__link"><a 
href="/documentation/programming-guide/">Concepts</a></div>
+        <div class="footer__cols__col__link"><a 
href="/documentation/pipelines/design-your-pipeline/">Pipelines</a></div>
+        <div class="footer__cols__col__link"><a 
href="/documentation/runners/capability-matrix/">Runners</a></div>
+      </div>
+      <div class="footer__cols__col footer__cols__col--md">
+        <div class="footer__cols__col__title">Community</div>
+        <div class="footer__cols__col__link"><a 
href="/contribute/">Contribute</a></div>
+        <div class="footer__cols__col__link"><a 
href="/contribute/team/">Team</a></div>
+        <div class="footer__cols__col__link"><a 
href="/contribute/presentation-materials/">Media</a></div>
+      </div>
+      <div class="footer__cols__col footer__cols__col--md">
+        <div class="footer__cols__col__title">Resources</div>
+        <div class="footer__cols__col__link"><a href="/blog/">Blog</a></div>
+        <div class="footer__cols__col__link"><a 
href="/get-started/support/">Support</a></div>
+        <div class="footer__cols__col__link"><a 
href="https://github.com/apache/beam";>GitHub</a></div>
+      </div>
+    </div>
+  </div>
+  <div class="footer__bottom">
+    &copy;
+    <a href="http://www.apache.org";>The Apache Software Foundation</a>
+    | <a href="/privacy_policy">Privacy Policy</a>
+    | <a href="/feed.xml">RSS Feed</a>
+    <br><br>
+    Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are
+    either registered trademarks or trademarks of The Apache Software
+    Foundation. All other products or name brands are trademarks of their
+    respective holders, including The Apache Software Foundation.
+  </div>
+</footer>
+
+  </body>
+</html>
diff --git a/content/blog/index.html b/content/blog/index.html
index a68c0ec..c52c16e 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -86,6 +86,23 @@
 <p>This is the blog for the Apache Beam project. This blog contains news and 
updates
 for the project.</p>
 
+<h3 id="a-classpost-link-hrefblog20180219beam-230htmlapache-beam-230a"><a 
class="post-link" href="/blog/2018/02/19/beam-2.3.0.html">Apache Beam 
2.3.0</a></h3>
+<p><i>Feb 19, 2018 •  Ismaël Mejía [<a 
href="https://twitter.com/iemejia";>@iemejia</a>]
+</i></p>
+
+<p>We are glad to present the new 2.3.0 release of Beam. This release includes
+multiple fixes and new functionalities.</p>
+
+<!-- Render a "read more" button if the post is longer than the excerpt -->
+
+<p>
+<a class="btn btn-default btn-sm" href="/blog/2018/02/19/beam-2.3.0.html" 
role="button">
+Read more&nbsp;<span class="glyphicon glyphicon-menu-right" 
aria-hidden="true"></span>
+</a>
+</p>
+
+<hr />
+
 <h3 
id="a-classpost-link-hrefblog20180109beam-a-look-backhtmlapache-beam-a-look-back-at-2017a"><a
 class="post-link" href="/blog/2018/01/09/beam-a-look-back.html">Apache Beam: A 
Look Back at 2017</a></h3>
 <p><i>Jan 9, 2018 •  Anand Iyer  &amp; Jean-Baptiste Onofré [<a 
href="https://twitter.com/jbonofre";>@jbonofre</a>]
 </i></p>
diff --git a/content/feed.xml b/content/feed.xml
index 7adc227..e0112c6 100644
--- a/content/feed.xml
+++ b/content/feed.xml
@@ -9,6 +9,103 @@
     <generator>Jekyll v3.2.0</generator>
     
       <item>
+        <title>Apache Beam 2.3.0</title>
+        <description>&lt;p&gt;We are glad to present the new 2.3.0 release of 
Beam. This release includes
+multiple fixes and new functionalities. &lt;!--more--&gt; For more information
+please check the detailed release notes.&lt;/p&gt;
+
+&lt;h1 id=&quot;new-features--improvements&quot;&gt;New Features / 
Improvements&lt;/h1&gt;
+
+&lt;h2 id=&quot;beam-moves-to-java-8&quot;&gt;Beam moves to Java 8&lt;/h2&gt;
+
+&lt;p&gt;The supported version of Java for Beam is now Java 8. The code and 
examples have
+been refactored to use multiple of the advantages of the language, e.g. 
lambdas,
+streams, improved type inference, etc.&lt;/p&gt;
+
+&lt;h2 id=&quot;spark-runner-is-now-based-on-spark-2x&quot;&gt;Spark runner is 
now based on Spark 2.x&lt;/h2&gt;
+
+&lt;p&gt;Spark runner moves forward into the Spark 2.x development line, this 
would allow
+to benefit of improved performance, as well as open the runner for future
+compatibility with the Structured Streaming APIs. Notice that support for Spark
+1.x is finished with this release.&lt;/p&gt;
+
+&lt;h2 id=&quot;amazon-web-services-s3-filesystem-support&quot;&gt;Amazon Web 
Services S3 Filesystem support&lt;/h2&gt;
+
+&lt;p&gt;Beam already supported AWS S3 via HadoopFileSystem, but this version 
brings a
+native implementation with the corresponding performance advantages of the S3
+filesystem.&lt;/p&gt;
+
+&lt;h2 id=&quot;general-purpose-writing-to-files&quot;&gt;General-purpose 
writing to files&lt;/h2&gt;
+
+&lt;p&gt;This release contains a new transform, FileIO.write() / 
writeDynamic() that
+implements a general-purpose fluent and Java8-friendly API for writing to files
+using a FileIO.Sink. This API has similar capabilities to DynamicDestinations
+APIs from Beam 2.2 but is much easier to use and extend. The 
DynamicDestinations
+APIs for writing to files are deprecated by it, as is FileBasedSink.&lt;/p&gt;
+
+&lt;h2 id=&quot;splittable-dofn-support-on-the-python-sdk&quot;&gt;Splittable 
DoFn support on the Python SDK&lt;/h2&gt;
+
+&lt;p&gt;This release adds the Splittable DoFn API for Python SDK and adds 
Splittable
+DoFn support for Python streaming DirectRunner.&lt;/p&gt;
+
+&lt;h2 id=&quot;portability&quot;&gt;Portability&lt;/h2&gt;
+
+&lt;p&gt;Progress continues to being able to execute Python on runners other 
then Google
+Cloud Dataflow and the Go SDK on any runner.&lt;/p&gt;
+
+&lt;h1 id=&quot;miscellaneous-fixes&quot;&gt;Miscellaneous Fixes&lt;/h1&gt;
+
+&lt;h2 id=&quot;sdks&quot;&gt;SDKs&lt;/h2&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;MapElements and FlatMapElements support using side inputs using 
the new
+interface Contextful.Fn. For library authors, this interface is the
+recommended choice for user-code callbacks that may use side inputs.&lt;/li&gt;
+  &lt;li&gt;Introduces the family of Reify transforms for converting between 
explicit and
+implicit representations of various Beam entities.&lt;/li&gt;
+  &lt;li&gt;Introduces two transforms for approximate sketching of data: 
Count-Min Sketch
+(approximate element frequency estimation) and HyperLogLog (approximate
+cardinality estimation).&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;h2 id=&quot;runners&quot;&gt;Runners&lt;/h2&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;Staging files on Dataflow shows progress&lt;/li&gt;
+  &lt;li&gt;Flink runner is based now on Flink version 1.4.0&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;h2 id=&quot;ios&quot;&gt;IOs&lt;/h2&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;BigtableIO now supports ValueProvider configuration&lt;/li&gt;
+  &lt;li&gt;BigQueryIO supports writing bounded collections to tables with 
partition
+decorators&lt;/li&gt;
+  &lt;li&gt;KafkaIO moves to version 1.0 (it is still backwards compatible 
with versions &amp;gt;= 0.9.x.x)&lt;/li&gt;
+  &lt;li&gt;Added IO source for VCF files (Python)&lt;/li&gt;
+  &lt;li&gt;Added support for backoff on deadlocks in JdbcIO.write() and 
connection
+improvement&lt;/li&gt;
+  &lt;li&gt;Improved performance of KinesisIO.read()&lt;/li&gt;
+  &lt;li&gt;Many improvements to TikaIO&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;h1 id=&quot;list-of-contributors&quot;&gt;List of Contributors&lt;/h1&gt;
+
+&lt;p&gt;According to git shortlog, the following 80 people contributed to the 
2.3.0 release. Thank you to all contributors!&lt;/p&gt;
+
+&lt;p&gt;Ahmet Altay, Alan Myrvold, Alex Amato, Alexey Romanenko, Ankur 
Goenka, Anton Kedin, Arnaud Fournier, Alexey Romanenko, Asha Rostamianfar, Ben 
Chambers, Ben Sidhom, Bill Neubauer, Brian Foo, cclauss, Chamikara Jayalath, 
Charles Chen, Colm O hEigeartaigh, Daniel Oliveira, Dariusz Aniszewski, David 
Cavazos, David Sabater, David Sabater Dinter, Dawid Wysakowicz, Dmytro Ivanov, 
Etienne Chauchot, Eugene Kirpichov, Exprosed, Grzegorz Kołakowski, Henning 
Rohde, Holden Karau, Huygaa Bats [...]
+
+</description>
+        <pubDate>Mon, 19 Feb 2018 00:00:01 -0800</pubDate>
+        <link>https://beam.apache.org/blog/2018/02/19/beam-2.3.0.html</link>
+        <guid 
isPermaLink="true">https://beam.apache.org/blog/2018/02/19/beam-2.3.0.html</guid>
+        
+        
+        <category>blog</category>
+        
+      </item>
+    
+      <item>
         <title>Apache Beam: A Look Back at 2017</title>
         <description>&lt;p&gt;On January 10, 2017, Apache Beam got &lt;a 
href=&quot;/blog/2017/01/10/beam-graduates.html&quot;&gt;promoted&lt;/a&gt;
 as a Top-Level Apache Software Foundation project. It was an important 
milestone
@@ -2153,318 +2250,5 @@ tracker, pull requests, and events.&lt;/p&gt;
         
       </item>
     
-      <item>
-        <title>Testing Unbounded Pipelines in Apache Beam</title>
-        <description>&lt;p&gt;The Beam Programming Model unifies writing 
pipelines for Batch and Streaming
-pipelines. We’ve recently introduced a new PTransform to write tests for
-pipelines that will be run over unbounded datasets and must handle out-of-order
-and delayed data.
-&lt;!--more--&gt;&lt;/p&gt;
-
-&lt;p&gt;Watermarks, Windows and Triggers form a core part of the Beam 
programming model
-– they respectively determine how your data are grouped, when your input is
-complete, and when to produce results. This is true for all pipelines,
-regardless of if they are processing bounded or unbounded inputs. If you’re not
-familiar with watermarks, windowing, and triggering in the Beam model,
-&lt;a 
href=&quot;https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101&quot;&gt;Streaming
 101&lt;/a&gt;
-and &lt;a 
href=&quot;https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102&quot;&gt;Streaming
 102&lt;/a&gt;
-are an excellent place to get started. A key takeaway from
-these articles: in realistic streaming scenarios with intermittent failures and
-disconnected users, data can arrive out of order or be delayed. Beam’s
-primitives provide a way for users to perform useful, powerful, and correct
-computations in spite of these challenges.&lt;/p&gt;
-
-&lt;p&gt;As Beam pipeline authors, we need comprehensive tests that cover 
crucial
-failure scenarios and corner cases to gain real confidence that a pipeline is
-ready for production. The existing testing infrastructure within the Beam SDKs
-permits tests to be written which examine the contents of a Pipeline at
-execution time. However, writing unit tests for pipelines that may receive
-late data or trigger multiple times has historically ranged from complex to
-not possible, as pipelines that read from unbounded sources do not shut down
-without external intervention, while pipelines that read from bounded sources
-exclusively cannot test behavior with late data nor most speculative triggers.
-Without additional tools, pipelines that use custom triggers and handle
-out-of-order data could not be easily tested.&lt;/p&gt;
-
-&lt;p&gt;This blog post introduces our new framework for writing tests for 
pipelines that
-handle delayed and out-of-order data in the context of the LeaderBoard pipeline
-from the Mobile Gaming example series.&lt;/p&gt;
-
-&lt;h2 id=&quot;leaderboard-and-the-mobile-gaming-example&quot;&gt;LeaderBoard 
and the Mobile Gaming Example&lt;/h2&gt;
-
-&lt;p&gt;&lt;a 
href=&quot;https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177&quot;&gt;LeaderBoard&lt;/a&gt;
-is part of the &lt;a 
href=&quot;https://github.com/apache/beam/tree/master/examples/java/src/main/java/org/apache/beam/examples/complete/game&quot;&gt;Beam
 mobile gaming examples&lt;/a&gt;
-(and &lt;a 
href=&quot;/get-started/mobile-gaming-example/&quot;&gt;walkthroughs&lt;/a&gt;)
-which produces a continuous accounting of user and team scores. User scores are
-calculated over the lifetime of the program, while team scores are calculated
-within fixed windows with a default duration of one hour. The LeaderBoard
-pipeline produces speculative and late panes as appropriate, based on the
-configured triggering and allowed lateness of the pipeline. The expected 
outputs
-of the LeaderBoard pipeline vary depending on when elements arrive in relation
-to the watermark and the progress of processing time, which could not 
previously
-be controlled within a test.&lt;/p&gt;
-
-&lt;h2 
id=&quot;writing-deterministic-tests-to-emulate-nondeterminism&quot;&gt;Writing 
Deterministic Tests to Emulate Nondeterminism&lt;/h2&gt;
-
-&lt;p&gt;The Beam testing infrastructure provides the
-&lt;a 
href=&quot;/documentation/sdks/javadoc/2.3.0/org/apache/beam/sdk/testing/PAssert.html&quot;&gt;PAssert&lt;/a&gt;
-methods, which assert properties about the contents of a PCollection from 
within
-a pipeline. We have expanded this infrastructure to include
-&lt;a 
href=&quot;https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestStream.java&quot;&gt;TestStream&lt;/a&gt;,
-which is a PTransform that performs a series of events, consisting of adding
-additional elements to a pipeline, advancing the watermark of the TestStream,
-and advancing the pipeline processing time clock. TestStream permits tests 
which
-observe the effects of triggers on the output a pipeline produces.&lt;/p&gt;
-
-&lt;p&gt;While executing a pipeline that reads from a TestStream, the read 
waits for all
-of the consequences of each event to complete before continuing on to the next
-event, ensuring that when processing time advances, triggers that are based on
-processing time fire as appropriate. With this transform, the effect of
-triggering and allowed lateness can be observed on a pipeline, including
-reactions to speculative and late panes and dropped data.&lt;/p&gt;
-
-&lt;h2 id=&quot;element-timings&quot;&gt;Element Timings&lt;/h2&gt;
-
-&lt;p&gt;Elements arrive either before, with, or after the watermark, which 
categorizes
-them into the “early”, “on-time”, and “late” divisions. “Late” elements can be
-further subdivided into “unobservably”, “observably”, and “droppably” late,
-depending on the window to which they are assigned and the maximum allowed
-lateness, as specified by the windowing strategy. Elements that arrive with
-these timings are emitted into panes, which can be “EARLY”, “ON-TIME”, or
-“LATE”, depending on the position of the watermark when the pane was 
emitted.&lt;/p&gt;
-
-&lt;p&gt;Using TestStream, we can write tests that demonstrate that 
speculative panes are
-output after their trigger condition is met, that the advancing of the 
watermark
-causes the on-time pane to be produced, and that late-arriving data produces
-refinements when it arrives before the maximum allowed lateness, and is dropped
-after.&lt;/p&gt;
-
-&lt;p&gt;The following examples demonstrate how you can use TestStream to 
provide a
-sequence of events to the Pipeline, where the arrival of elements is 
interspersed
-with updates to the watermark and the advance of processing time. Each of these
-events runs to completion before additional events occur.&lt;/p&gt;
-
-&lt;p&gt;In the diagrams, the time at which events occurred in “real” (event) 
time
-progresses as the graph moves to the right. The time at which the pipeline
-receives them progresses as the graph goes upwards. The watermark is 
represented
-by the squiggly red line, and each starburst is the firing of a trigger and the
-associated pane.&lt;/p&gt;
-
-&lt;p&gt;&lt;img class=&quot;center-block&quot; 
src=&quot;/images/blog/test-stream/elements-all-on-time.png&quot; 
alt=&quot;Elements on the Event and Processing time axes, with the Watermark 
and produced panes&quot; width=&quot;442&quot; /&gt;&lt;/p&gt;
-
-&lt;h3 id=&quot;everything-arrives-on-time&quot;&gt;Everything arrives 
on-time&lt;/h3&gt;
-
-&lt;p&gt;For example, if we create a TestStream where all the data arrives 
before the
-watermark and provide the result PCollection as input to the 
CalculateTeamScores
-PTransform:&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;infos&lt;/span&gt; &lt;span 
class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;sp [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;sky&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span class=&q [...]
- &lt;span class=&quot;err&quot;&gt;                &lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;navy&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span cl [...]
- &lt;span class=&quot;err&quot;&gt;                &lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;navy&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span cl [...]
- &lt;span class=&quot;err&quot;&gt;   &lt;/span&gt;&lt;span 
class=&quot;c1&quot;&gt;// Move the watermark past the end the end of the 
window&lt;/span&gt;
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkTo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Instant&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;0L&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/sp [...]
- &lt;span class=&quot;err&quot;&gt;                               
&lt;/span&gt;       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;Duration&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;standardMinutes&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span cl [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkToInfinity&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-
-&lt;span class=&quot;n&quot;&gt;PCollection&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;String&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Integer&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt; &lt;span class= [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;apply&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;CalculateTeamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;TEAM_WINDOW_DURATION&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;ALLOWED_LATENESS&lt;/span&gt;&lt;span clas [...]
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;we can then assert that the result PCollection contains elements that 
arrived:&lt;/p&gt;
-
-&lt;p&gt;&lt;img class=&quot;center-block&quot; 
src=&quot;/images/blog/test-stream/elements-all-on-time.png&quot; 
alt=&quot;Elements all arrive before the watermark, and are produced in the 
on-time pane&quot; width=&quot;442&quot; /&gt;&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c1&quot;&gt;// 
Only one value is emitted for the blue team&lt;/span&gt;
-&lt;span class=&quot;n&quot;&gt;PAssert&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;that&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;inWindow&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;window&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;containsInAnyOrder&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1 [...]
-&lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;run&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;h3 
id=&quot;some-elements-are-late-but-arrive-before-the-end-of-the-window&quot;&gt;Some
 elements are late, but arrive before the end of the window&lt;/h3&gt;
-
-&lt;p&gt;We can also add data to the TestStream after the watermark, but 
before the end
-of the window (shown below to the left of the red watermark), which 
demonstrates
-“unobservably late” data - that is, data that arrives late, but is promoted by
-the system to be on time, as it arrives before the watermark passes the end of
-the window&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;infos&lt;/span&gt; &lt;span 
class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;sp [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;sky&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span class=&q [...]
- &lt;span class=&quot;err&quot;&gt;       &lt;/span&gt;         &lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;navy&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span cl [...]
- &lt;span class=&quot;err&quot;&gt;   &lt;/span&gt;&lt;span 
class=&quot;c1&quot;&gt;// Move the watermark up to &quot;near&quot; the end of 
the window&lt;/span&gt;
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkTo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Instant&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;0L&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/sp [...]
- &lt;span class=&quot;err&quot;&gt;               &lt;/span&gt;       &lt;span 
class=&quot;err&quot;&gt;                &lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;minus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;Duration&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;standardMinutes&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span c [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;sky&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span class=&q [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkToInfinity&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-
-&lt;span class=&quot;n&quot;&gt;PCollection&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;String&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Integer&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt; &lt;span class= [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;apply&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;CalculateTeamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;TEAM_WINDOW_DURATION&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;ALLOWED_LATENESS&lt;/span&gt;&lt;span clas [...]
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;&lt;img class=&quot;center-block&quot; 
src=&quot;/images/blog/test-stream/elements-unobservably-late.png&quot; 
alt=&quot;An element arrives late, but before the watermark passes the end of 
the window, and is produced in the on-time pane&quot; width=&quot;442&quot; 
/&gt;&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c1&quot;&gt;// 
Only one value is emitted for the blue team&lt;/span&gt;
-&lt;span class=&quot;n&quot;&gt;PAssert&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;that&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;inWindow&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;window&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;containsInAnyOrder&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1 [...]
-&lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;run&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;h3 
id=&quot;elements-are-late-and-arrive-after-the-end-of-the-window&quot;&gt;Elements
 are late, and arrive after the end of the window&lt;/h3&gt;
-
-&lt;p&gt;By advancing the watermark farther in time before adding the late 
data, we can
-demonstrate the triggering behavior that causes the system to emit an on-time
-pane, and then after the late data arrives, a pane that refines the 
result.&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;infos&lt;/span&gt; &lt;span 
class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;sp [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;sky&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span class=&q [...]
-       &lt;span class=&quot;err&quot;&gt;          &lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;navy&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span cl [...]
-&lt;span class=&quot;err&quot;&gt; &lt;/span&gt; &lt;span 
class=&quot;err&quot;&gt;  &lt;/span&gt;&lt;span class=&quot;c1&quot;&gt;// 
Move the watermark up to &quot;near&quot; the end of the window&lt;/span&gt;
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkTo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Instant&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;0L&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/sp [...]
-&lt;span class=&quot;err&quot;&gt;                                
&lt;/span&gt;       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;minus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;Duration&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;standardMinutes&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span c [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;sky&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span class=&q [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkToInfinity&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-
-&lt;span class=&quot;n&quot;&gt;PCollection&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;String&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Integer&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt; &lt;span class= [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;apply&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;CalculateTeamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;TEAM_WINDOW_DURATION&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;ALLOWED_LATENESS&lt;/span&gt;&lt;span clas [...]
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;&lt;img class=&quot;center-block&quot; 
src=&quot;/images/blog/test-stream/elements-observably-late.png&quot; 
alt=&quot;Elements all arrive before the watermark, and are produced in the 
on-time pane&quot; width=&quot;442&quot; /&gt;&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c1&quot;&gt;// 
An on-time pane is emitted with the events that arrived before the window 
closed&lt;/span&gt;
-&lt;span class=&quot;n&quot;&gt;PAssert&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;that&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;inOnTimePane&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;window&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;containsInAnyOrder&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;6 [...]
-&lt;span class=&quot;c1&quot;&gt;// The final pane contains the late 
refinement&lt;/span&gt;
-&lt;span class=&quot;n&quot;&gt;PAssert&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;that&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;inFinalPane&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;window&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;containsInAnyOrder&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1 [...]
-&lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;run&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;h3 
id=&quot;elements-are-late-and-after-the-end-of-the-window-plus-the-allowed-lateness&quot;&gt;Elements
 are late, and after the end of the window plus the allowed lateness&lt;/h3&gt;
-
-&lt;p&gt;If we push the watermark even further into the future, beyond the 
maximum
-configured allowed lateness, we can demonstrate that the late element is 
dropped
-by the system.&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;infos&lt;/span&gt; &lt;span 
class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;sp [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;sky&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span class=&q [...]
-        &lt;span class=&quot;err&quot;&gt;         &lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;navy&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span cl [...]
-&lt;span class=&quot;err&quot;&gt;    &lt;/span&gt;&lt;span 
class=&quot;c1&quot;&gt;// Move the watermark up to &quot;near&quot; the end of 
the window&lt;/span&gt;
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkTo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Instant&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/spa [...]
- &lt;span class=&quot;err&quot;&gt;                                        
&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;ALLOWED_LATENESS&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
- &lt;span class=&quot;err&quot;&gt;                                        
&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;Duration&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;standardMinutes&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span  [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;
-&lt;span class=&quot;err&quot;&gt;                     &lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;sky&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt;
-&lt;span class=&quot;err&quot;&gt;                     &lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt;
-&lt;span class=&quot;err&quot;&gt;                     &lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;12&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt;
-&lt;span class=&quot;err&quot;&gt;                     &lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Instant&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;plus&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;TEAM_WINDOW_DURATION&lt;/span&gt;&lt;span class=&quot;o 
[...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkToInfinity&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-
-&lt;span class=&quot;n&quot;&gt;PCollection&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;String&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Integer&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt; &lt;span class= [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;apply&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;CalculateTeamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;TEAM_WINDOW_DURATION&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;ALLOWED_LATENESS&lt;/span&gt;&lt;span clas [...]
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;&lt;img class=&quot;center-block&quot; 
src=&quot;/images/blog/test-stream/elements-droppably-late.png&quot; 
alt=&quot;Elements all arrive before the watermark, and are produced in the 
on-time pane&quot; width=&quot;442&quot; /&gt;&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c1&quot;&gt;// 
An on-time pane is emitted with the events that arrived before the window 
closed&lt;/span&gt;
-&lt;span class=&quot;n&quot;&gt;PAssert&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;that&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;teamScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;inWindow&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;window&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;containsInAnyOrder&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;blue&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;6 [...]
-
-&lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;run&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;h3 
id=&quot;elements-arrive-before-the-end-of-the-window-and-some-processing-time-passes&quot;&gt;Elements
 arrive before the end of the window, and some processing time passes&lt;/h3&gt;
-&lt;p&gt;Using additional methods, we can demonstrate the behavior of 
speculative
-triggers by advancing the processing time of the TestStream. If we add elements
-to an input PCollection, occasionally advancing the processing time clock, and
-apply &lt;code 
class=&quot;highlighter-rouge&quot;&gt;CalculateUserScores&lt;/code&gt;&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span 
class=&quot;n&quot;&gt;TestStream&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;create&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;AvroCoder&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;( [...]
- &lt;span class=&quot;err&quot;&gt;   &lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;scarlet&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot [...]
- &lt;span class=&quot;err&quot;&gt;              &lt;/span&gt;  &lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;scarlet&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;red&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span  [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceProcessingTime&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;Duration&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;standardMinutes&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;12&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;))&lt;/span&gt;
- &lt;span class=&quot;err&quot;&gt;   &lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;addElements&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;oxblood&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s&quot [...]
- &lt;span class=&quot;err&quot;&gt;              &lt;/span&gt;  &lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;GameActionInfo&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;scarlet&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;s&quot;&gt;&quot;red&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;4&lt;/span&gt;&lt;span  [...]
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceProcessingTime&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;Duration&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;standardMinutes&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;mi&quot;&gt;15&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;))&lt;/span&gt;
-    &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;advanceWatermarkToInfinity&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-
-&lt;span class=&quot;n&quot;&gt;PCollection&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;String&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;Integer&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span 
class=&quot;n&quot;&gt;userScores&lt;/span&gt; &lt;span class= [...]
- &lt;span class=&quot;err&quot;&gt;   &lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;apply&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;infos&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;apply&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;k&quot;&gt;new&lt;/span&gt; &lt;span  [...]
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;&lt;img class=&quot;center-block&quot; 
src=&quot;/images/blog/test-stream/elements-processing-speculative.png&quot; 
alt=&quot;Elements all arrive before the watermark, and are produced in the 
on-time pane&quot; width=&quot;442&quot; /&gt;&lt;/p&gt;
-
-&lt;div class=&quot;language-java highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span 
class=&quot;n&quot;&gt;PAssert&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;that&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;userScores&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;)&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;inEarlyGlobalWindowPanes&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;()&lt;/span&gt;
-       &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;containsInAnyOrder&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;scarlet&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&g [...]
-   &lt;span class=&quot;err&quot;&gt;                        
&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;scarlet&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;9&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;),&lt;/span&gt;
-                           &lt;span 
class=&quot;n&quot;&gt;KV&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;of&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span 
class=&quot;s&quot;&gt;&quot;oxblood&quot;&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span 
class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;));&lt;/span&gt;
-
-&lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span 
class=&quot;na&quot;&gt;run&lt;/span&gt;&lt;span 
class=&quot;o&quot;&gt;();&lt;/span&gt;
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;h2 id=&quot;teststream---under-the-hood&quot;&gt;TestStream - Under the 
Hood&lt;/h2&gt;
-
-&lt;p&gt;TestStream relies on a pipeline concept we’ve introduced, called 
quiescence, to
-utilize the existing runner infrastructure while providing guarantees about 
when
-a root transform will called by the runner. This consists of properties about
-pending elements and triggers, namely:&lt;/p&gt;
-
-&lt;ul&gt;
-  &lt;li&gt;No trigger is permitted to fire but has not fired&lt;/li&gt;
-  &lt;li&gt;All elements are either buffered in state or cannot progress until 
a side input becomes available&lt;/li&gt;
-&lt;/ul&gt;
-
-&lt;p&gt;Simplified, this means that, in the absence of an advancement in input
-watermarks or processing time, or additional elements being added to the
-pipeline, the pipeline will not make progress. Whenever the TestStream 
PTransform
-performs an action, the runner must not reinvoke the same instance until the
-pipeline has quiesced. This ensures that the events specified by TestStream
-happen “in-order”, which ensures that input watermarks and the system clock do
-not advance ahead of the elements they hoped to hold up.&lt;/p&gt;
-
-&lt;p&gt;The DirectRunner has been modified to use quiescence as the signal 
that it
-should add more work to the Pipeline, and the implementation of TestStream in
-that runner uses this fact to perform a single output per event. The 
DirectRunner
-implementation also directly controls the runner’s system clock, ensuring that
-tests will complete promptly even if there is a multi-minute processing time
-trigger located within the pipeline.&lt;/p&gt;
-
-&lt;p&gt;The TestStream transform is supported in the DirectRunner. For most 
users, tests
-written using TestPipeline and PAsserts will automatically function while using
-TestStream.&lt;/p&gt;
-
-&lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;
-
-&lt;p&gt;The addition of TestStream alongside window and pane-specific 
matchers in PAssert
-has enabled the testing of Pipelines which produce speculative and late panes.
-This permits tests for all styles of pipeline to be expressed directly within 
the
-Java SDK. If you have questions or comments, we’d love to hear them on the
-&lt;a href=&quot;/get-started/support/&quot;&gt;mailing 
lists&lt;/a&gt;.&lt;/p&gt;
-</description>
-        <pubDate>Thu, 20 Oct 2016 11:00:00 -0700</pubDate>
-        <link>https://beam.apache.org/blog/2016/10/20/test-stream.html</link>
-        <guid 
isPermaLink="true">https://beam.apache.org/blog/2016/10/20/test-stream.html</guid>
-        
-        
-        <category>blog</category>
-        
-      </item>
-    
   </channel>
 </rss>
diff --git a/content/index.html b/content/index.html
index ba11ebf..3d48f95 100644
--- a/content/index.html
+++ b/content/index.html
@@ -109,6 +109,11 @@
           </div>
           <div class="hero__blog__cards">
             
+            <a class="hero__blog__cards__card" 
href="/blog/2018/02/19/beam-2.3.0.html">
+              <div class="hero__blog__cards__card__title">Apache Beam 
2.3.0</div>
+              <div class="hero__blog__cards__card__date">Feb 19, 2018</div>
+            </a>
+            
             <a class="hero__blog__cards__card" 
href="/blog/2018/01/09/beam-a-look-back.html">
               <div class="hero__blog__cards__card__title">Apache Beam: A Look 
Back at 2017</div>
               <div class="hero__blog__cards__card__date">Jan 9, 2018</div>
@@ -119,11 +124,6 @@
               <div class="hero__blog__cards__card__date">Aug 28, 2017</div>
             </a>
             
-            <a class="hero__blog__cards__card" 
href="/blog/2017/08/16/splittable-do-fn.html">
-              <div class="hero__blog__cards__card__title">Powerful and modular 
IO connectors with Splittable DoFn in Apache Beam</div>
-              <div class="hero__blog__cards__card__date">Aug 16, 2017</div>
-            </a>
-            
           </div>
         </div>
       </div>

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

Reply via email to