This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 52db1c3 Publishing website 2020/10/08 00:03:51 at commit f338284
52db1c3 is described below
commit 52db1c38dd087035a40b31cf9c91522cd016df76
Author: jenkins <[email protected]>
AuthorDate: Thu Oct 8 00:03:51 2020 +0000
Publishing website 2020/10/08 00:03:51 at commit f338284
---
website/generated-content/documentation/index.xml | 94 ++++++++++++++++------
.../documentation/programming-guide/index.html | 72 ++++++++++++-----
website/generated-content/sitemap.xml | 2 +-
3 files changed, 125 insertions(+), 43 deletions(-)
diff --git a/website/generated-content/documentation/index.xml
b/website/generated-content/documentation/index.xml
index 6aab1a5..cb772ab 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -3198,7 +3198,7 @@ elements from the input collection).</li>
<li>Once you output a value using
<code>OutputReceiver.output()</code> you should not modify
that value in any way.</li>
</ul></p>
-<p class="language-python"><ul>
+<p class="language-py"><ul>
<li>You should not in any way modify the <code>element</code>
argument provided to the
<code>process</code> method, or any side inputs.</li>
<li>Once you output a value using <code>yield</code> or
<code>return</code>, you should not modify
@@ -4529,6 +4529,7 @@ of the Beam SDK being used. This allows Beam users to
continue using native type
having Beam understand their element schemas.</p>
<p class="language-java">In Java you could use the following set of classes
to represent the purchase schema. Beam will automatically
infer the correct schema based on the members of the class.</p>
+<p class="language-py">In Python you can use the following set of classes
to represent the purchase schema. Beam will automatically infer the correct
schema based on the members of the class.</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"><span
class="nd">@DefaultSchema</span><span class="o">(</span><span
class="n">JavaBeanSchema</span><span class="o">.</span><span
class="na">class</span><span class="o">)</span>
<span class="kd">public</span> <span class="kd">class</span>
<span class="nc">Purchase</span> <span class="o">{</span>
@@ -4567,7 +4568,25 @@ infer the correct schema based on the members of the
class.</p>
<span class="o">}</span>
<span class="o">}</span></code></pre></div>
</div>
-<p>Using JavaBean classes as above is one way to map a schema to Java
classes. However multiple Java classes might have
+<div class=language-py>
+<div class="highlight"><pre class="chroma"><code class="language-py"
data-lang="py"><span class="kn">import</span> <span
class="nn">typing</span>
+<span class="k">class</span> <span
class="nc">Purchase</span><span class="p">(</span><span
class="n">typing</span><span class="o">.</span><span
class="n">NamedTuple</span><span class="p">):</span>
+<span class="n">user_id</span><span class="p">:</span> <span
class="nb">str</span> <span class="c1"># The id of the user who made the
purchase.</span>
+<span class="n">item_id</span><span class="p">:</span> <span
class="nb">int</span> <span class="c1"># The identifier of the item that
was purchased.</span>
+<span class="n">shipping_address</span><span class="p">:</span>
<span class="n">ShippingAddress</span> <span class="c1"># The shipping
address, a nested type.</span>
+<span class="n">cost_cents</span><span class="p">:</span> <span
class="nb">int</span> <span class="c1"># The cost of the item</span>
+<span class="n">transactions</span><span class="p">:</span>
<span class="n">typing</span><span class="o">.</span><span
class="n">Sequence</span><span class="p">[</span><span
class="n">Transaction</span><span class="p">]</span> <span
class="c1"># The transactions that paid for this purchase (a list, since the
purchase might be spread out over multiple credit cards).</span>
+<span class="k">class</span> <span
class="nc">ShippingAddress</span><span class="p">(</span><span
class="n">typing</span><span class="o">.</span><span
class="n">NamedTuple</span><span class="p">):</span>
+<span class="n">street_address</span><span class="p">:</span>
<span class="nb">str</span>
+<span class="n">city</span><span class="p">:</span> <span
class="nb">str</span>
+<span class="n">state</span><span class="p">:</span> <span
class="n">typing</span><span class="o">.</span><span
class="n">Optional</span><span class="p">[</span><span
class="nb">str</span><span class="p">]</span>
+<span class="n">country</span><span class="p">:</span> <span
class="nb">str</span>
+<span class="n">postal_code</span><span class="p">:</span>
<span class="nb">str</span>
+<span class="k">class</span> <span
class="nc">Transaction</span><span class="p">(</span><span
class="n">typing</span><span class="o">.</span><span
class="n">NamedTuple</span><span class="p">):</span>
+<span class="n">bank</span><span class="p">:</span> <span
class="nb">str</span>
+<span class="n">purchase_amount</span><span class="p">:</span>
<span class="nb">float</span></code></pre></div>
+</div>
+<p class="language-java">Using JavaBean classes as above is one way to map
a schema to Java classes. However multiple Java classes might have
the same schema, in which case the different Java types can often be used
interchangeably. Beam will add implicit
conversions between types that have matching schemas. For example, the above
<code>Transaction</code> class has the same schema as the following
class:</p>
@@ -4578,12 +4597,12 @@ conversions between types that have matching schemas.
For example, the above
<span class="kd">public</span> <span class="kt">double</span>
<span class="n">purchaseAmount</span><span class="o">;</span>
<span class="o">}</span></code></pre></div>
</div>
-<p>So if we had two <code>PCollection</code>s as follows</p>
+<p class="language-java">So if we had two <code>PCollection</code>s
as follows</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"><span
class="n">PCollection</span><span class="o">&lt;</span><span
class="n">Transaction</span><span class="o">&gt;</span> <span
class="n">transactionBeans</span> <span class="o">=</span> <span
class="n">readTransactionsAsJavaBean</span><span class="o">();</span>
<span class="n">PCollection</span><span
class="o">&lt;</span><span
class="n">TransactionPojos</span><span class="o">&gt;</span>
<span class="n">transactionPojos</span> <span class="o">=</span>
<span class="n">readTransactionsAsPojo</span><span
class="o">();</span></code></pre></div>
</div>
-<p>Then these two <code>PCollection</code>s would have the same
schema, even though their Java types would be different. This means
+<p class="language-java">Then these two <code>PCollection</code>s
would have the same schema, even though their Java types would be different.
This means
for example the following two code snippets are valid:</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"><span
class="n">transactionBeans</span><span class="o">.</span><span
class="na">apply</span><span class="o">(</span><span
class="n">ParDo</span><span class="o">.</span><span
class="na">of</span><span class="o">(</span><span
class="k">new</span> <span class="n">DoFn</span><span
class="o">&lt;...&gt;()</span> <span cla [...]
@@ -4592,15 +4611,14 @@ for example the following two code snippets are
valid:</p>
<span class="o">}</span>
<span class="o">}));</span></code></pre></div>
</div>
-<p>and
+<p class="language-java">and</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"><span
class="n">transactionPojos</span><span class="o">.</span><span
class="na">apply</span><span class="o">(</span><span
class="n">ParDo</span><span class="o">.</span><span
class="na">of</span><span class="o">(</span><span
class="k">new</span> <span class="n">DoFn</span><span
class="o">&lt;...&gt;()</span> <span cla [...]
<span class="nd">@ProcessElement</span> <span
class="kd">public</span> <span class="kt">void</span> <span
class="nf">process</span><span class="o">(</span><span
class="nd">@Element</span> <span class="n">Transaction</span> <span
class="n">row</span><span class="o">)</span> <span
class="o">{</span>
<span class="o">}</span>
<span class="o">}));</span></code></pre></div>
</div>
-</p>
-<p>Even though the in both cases the <code>@Element</code> parameter
differs from the the <code>PCollection</code>'s Java type, since the
+<p class="language-java">Even though the in both cases the
<code>@Element</code> parameter differs from the the
<code>PCollection</code>'s Java type, since the
schemas are the same Beam will automatically make the conversion. The built-in
<code>Convert</code> transform can also be used
to translate between Java types of equivalent schemas, as detailed
below.</p>
<h3 id="schema-definition">6.3. Schema definition</h3>
@@ -4771,13 +4789,13 @@ also be used.</p>
<p>In order to take advantage of schemas, your
<code>PCollection</code>s must have a schema attached to it. Often, the
source itself will attach a schema to the PCollection. For example, when using
<code>AvroIO</code> to read Avro files, the source can automatically
infer a Beam schema from the Avro schema and attach that to the Beam
<code>PCollection</code>. However not all sources produce schemas. In
addition, often Beam pipelines have intermediate stages and types [...]
<h4 id="inferring-schemas">6.5.1. Inferring schemas</h4>
<p class="language-java">Beam is able to infer schemas from a variety of
common Java types. The <code>@DefaultSchema</code> annotation can be used
to tell Beam to infer schemas from a specific type. The annotation takes a
<code>SchemaProvider</code> as an argument, and
<code>SchemaProvider</code> classes are already built in for common Java
types. The <code>SchemaRegistry</code> can also be invoked
programmatically for cases where it is not practical to annotat [...]
-<h5 id="java-pojos"><strong>Java POJOs</strong></h5>
-<p>A POJO (Plain Old Java Object) is a Java object that is not bound by any
restriction other than the Java Language
+<p class="language-java"><strong>Java POJOs</strong></p>
+<p class="language-java">A POJO (Plain Old Java Object) is a Java object
that is not bound by any restriction other than the Java Language
Specification. A POJO can contain member variables that are primitives, that
are other POJOs, or are collections maps or
arrays thereof. POJOs do not have to extend prespecified classes or extend any
specific interfaces.</p>
-<p>If a POJO class is annotated with
<code>@DefaultSchema(JavaFieldSchema.class)</code>, Beam will
automatically infer a schema for
+<p class="language-java">If a POJO class is annotated with
<code>@DefaultSchema(JavaFieldSchema.class)</code>, Beam will
automatically infer a schema for
this class. Nested classes are supported as are classes with
<code>List</code>, array, and <code>Map</code> fields.</p>
-<p>For example, annotating the following class tells Beam to infer a schema
from this POJO class and apply it to any
+<p class="language-java">For example, annotating the following class tells
Beam to infer a schema from this POJO class and apply it to any
<code>PCollection&lt;TransactionPojo&gt;</code>.</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"><span
class="nd">@DefaultSchema</span><span class="o">(</span><span
class="n">JavaFieldSchema</span><span class="o">.</span><span
class="na">class</span><span class="o">)</span>
@@ -4793,24 +4811,24 @@ this class. Nested classes are supported as are classes
with <code>List</c
<span class="c1">// Beam will automatically infer the correct schema for
this PCollection. No coder is needed as a result.
</span><span class="c1"></span><span
class="n">PCollection</span><span class="o">&lt;</span><span
class="n">TransactionPojo</span><span class="o">&gt;</span>
<span class="n">pojos</span> <span class="o">=</span> <span
class="n">readPojos</span><span
class="o">();</span></code></pre></div>
</div>
-<p>The <code>@SchemaCreate</code> annotation tells Beam that this
constructor can be used to create instances of TransactionPojo,
+<p class="language-java">The <code>@SchemaCreate</code> annotation
tells Beam that this constructor can be used to create instances of
TransactionPojo,
assuming that constructor parameters have the same names as the field names.
<code>@SchemaCreate</code> can also be used to annotate
static factory methods on the class, allowing the constructor to remain
private. If there is no <code>@SchemaCreate</code>
annotation then all the fields must be non-final and the class must have a
zero-argument constructor.</p>
-<p>There are a couple of other useful annotations that affect how Beam
infers schemas. By default the schema field names
+<p class="language-java">There are a couple of other useful annotations
that affect how Beam infers schemas. By default the schema field names
inferred will match that of the class field names. However
<code>@SchemaFieldName</code> can be used to specify a different name to
be used for the schema field. <code>@SchemaIgnore</code> can be used to
mark specific class fields as excluded from the inferred
schema. For example, it’s common to have ephemeral fields in a class that
should not be included in a schema
(e.g. caching the hash value to prevent expensive recomputation of the hash),
and <code>@SchemaIgnore</code> can be used to
exclude these fields. Note that ignored fields will not be included in the
encoding of these records.</p>
-<p>In some cases it is not convenient to annotate the POJO class, for
example if the POJO is in a different package that is
+<p class="language-java">In some cases it is not convenient to annotate the
POJO class, for example if the POJO is in a different package that is
not owned by the Beam pipeline author. In these cases the schema inference can
be triggered programmatically in
pipeline’s main function as follows:</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"> <span
class="n">pipeline</span><span class="o">.</span><span
class="na">getSchemaRegistry</span><span class="o">().</span><span
class="na">registerPOJO</span><span class="o">(</span><span
class="n">TransactionPOJO</span><span class="o">.</span><span
class="na">class</span><span
class="o">);</span></code></pre></div>
</div>
-<h5 id="java-beans"><strong>Java Beans</strong></h5>
-<p>Java Beans are a de-facto standard for creating reusable property
classes in Java. While the full
+<p class="language-java"><strong>Java Beans</strong></p>
+<p class="language-java">Java Beans are a de-facto standard for creating
reusable property classes in Java. While the full
standard has many characteristics, the key ones are that all properties are
accessed via getter and setter classes, and
the name format for these getters and setters is standardized. A Java Bean
class can be annotated with
<code>@DefaultSchema(JavaBeanSchema.class)</code> and Beam will
automatically infer a schema for this class. For example:</p>
@@ -4826,7 +4844,7 @@ the name format for these getters and setters is
standardized. A Java Bean class
<span class="c1">// Beam will automatically infer the correct schema for
this PCollection. No coder is needed as a result.
</span><span class="c1"></span><span
class="n">PCollection</span><span class="o">&lt;</span><span
class="n">TransactionBean</span><span class="o">&gt;</span>
<span class="n">beans</span> <span class="o">=</span> <span
class="n">readBeans</span><span
class="o">();</span></code></pre></div>
</div>
-<p>The <code>@SchemaCreate</code> annotation can be used to specify a
constructor or a static factory method, in which case the
+<p class="language-java">The <code>@SchemaCreate</code> annotation
can be used to specify a constructor or a static factory method, in which case
the
setters and zero-argument constructor can be omitted.</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"><span
class="nd">@DefaultSchema</span><span class="o">(</span><span
class="n">JavaBeanSchema</span><span class="o">.</span><span
class="na">class</span><span class="o">)</span>
@@ -4837,12 +4855,12 @@ setters and zero-argument constructor can be
omitted.</p>
<span class="kd">public</span> <span class="kt">double</span>
<span class="nf">getPurchaseAmount</span><span class="o">()</span>
<span class="o">{</span> <span class="err">…</span> <span
class="o">}</span>
<span class="o">}</span></code></pre></div>
</div>
-<p><code>@SchemaFieldName</code> and <code>@SchemaIgnore</code>
can be used to alter the schema inferred, just like with POJO classes.</p>
-<h5 id="autovalue"><strong>AutoValue</strong></h5>
-<p>Java value classes are notoriously difficult to generate correctly.
There is a lot of boilerplate you must create in
+<p class="language-java"><code>@SchemaFieldName</code> and
<code>@SchemaIgnore</code> can be used to alter the schema inferred, just
like with POJO classes.</p>
+<p class="language-java"><strong>AutoValue</strong></p>
+<p class="language-java">Java value classes are notoriously difficult to
generate correctly. There is a lot of boilerplate you must create in
order to properly implement a value class. AutoValue is a popular library for
easily generating such classes by
implementing a simple abstract base class.</p>
-<p>Beam can infer a schema from an AutoValue class. For example:</p>
+<p class="language-java">Beam can infer a schema from an AutoValue class.
For example:</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code
class="language-java" data-lang="java"><span
class="nd">@DefaultSchema</span><span class="o">(</span><span
class="n">AutoValueSchema</span><span class="o">.</span><span
class="na">class</span><span class="o">)</span>
<span class="nd">@AutoValue</span>
@@ -4851,9 +4869,39 @@ implementing a simple abstract base class.</p>
<span class="kd">public</span> <span class="kd">abstract</span>
<span class="kt">double</span> <span
class="nf">getPurchaseAmount</span><span class="o">();</span>
<span class="o">}</span></code></pre></div>
</div>
-<p>This is all that’s needed to generate a simple AutoValue class, and the
above <code>@DefaultSchema</code> annotation tells Beam to
+<p class="language-java">This is all that’s needed to generate a simple
AutoValue class, and the above <code>@DefaultSchema</code> annotation
tells Beam to
infer a schema from it. This also allows AutoValue elements to be used inside
of <code>PCollection</code>s.</p>
-<p><code>@SchemaFieldName</code> and <code>@SchemaIgnore</code>
can be used to alter the schema inferred.</p>
+<p class="language-java"><code>@SchemaFieldName</code> and
<code>@SchemaIgnore</code> can be used to alter the schema
inferred.</p>
+<p class="language-py">Beam has a few different mechanisms for inferring
schemas from Python code.</p>
+<p class="language-py"><strong>NamedTuple classes</strong></p>
+<p class="language-py">A <a
href="https://docs.python.org/3/library/typing.html#typing.NamedTuple";>NamedTuple</a>
+class is a Python class that wraps a <code>tuple</code>, assigning a
name to each element
+and restricting it to a particular type. Beam will automatically infer the
+schema for PCollections with <code>NamedTuple</code> output types. For
example:</p>
+<div class=language-py>
+<div class="highlight"><pre class="chroma"><code class="language-py"
data-lang="py"><span class="k">class</span> <span
class="nc">Transaction</span><span class="p">(</span><span
class="n">typing</span><span class="o">.</span><span
class="n">NamedTuple</span><span class="p">):</span>
+<span class="n">bank</span><span class="p">:</span> <span
class="nb">str</span>
+<span class="n">purchase_amount</span><span class="p">:</span>
<span class="nb">float</span>
+<span class="n">pc</span> <span class="o">=</span> <span
class="nb">input</span> <span class="o">|</span> <span
class="n">beam</span><span class="o">.</span><span
class="n">Map</span><span class="p">(</span><span
class="k">lambda</span> <span class="o">...</span><span
class="p">)</span><span class="o">.</span><span
class="n">with_output_types</span><span class="p">(</span><span
class="n">Transaction< [...]
+</div>
+<p class="language-py"><strong>beam.Row</strong></p>
+<p class="language-py">It&rsquo;s also possible to create ad-hoc schema
declarations with a simple lambda
+that returns instances of <code>beam.Row</code>:</p>
+<div class=language-py>
+<div class="highlight"><pre class="chroma"><code class="language-py"
data-lang="py"><span class="n">input_pc</span> <span
class="o">=</span> <span class="o">...</span> <span class="c1">#
{&#34;bank&#34;: ..., &#34;purchase_amount&#34;: ...}</span>
+<span class="n">output_pc</span> <span class="o">=</span> <span
class="n">input_pc</span> <span class="o">|</span> <span
class="n">beam</span><span class="o">.</span><span
class="n">Map</span><span class="p">(</span><span
class="k">lambda</span> <span class="n">item</span><span
class="p">:</span> <span class="n">beam</span><span
class="o">.</span><span class="n">Row</span><span
class="p">(</span>< [...]
+<span class="n">purchase_amount</span><span
class="o">=</span><span class="n">item</span><span
class="p">[</span><span
class="s2">&#34;purchase_amount&#34;</span><span
class="p">])</span></code></pre></div>
+</div>
+<p class="language-py">Note that this declaration doesn&rsquo;t include
any specific information about the
+types of the <code>bank</code> and <code>purchase_amount</code>
fields. Beam will attempt to infer
+type information, if it&rsquo;s unable to it will fall back to the generic
type
+<code>Any</code>. Sometimes this is not ideal, you can use casts to make
sure Beam
+correctly infers types with <code>beam.Row</code>:</p>
+<div class=language-py>
+<div class="highlight"><pre class="chroma"><code class="language-py"
data-lang="py"><span class="n">input_pc</span> <span
class="o">=</span> <span class="o">...</span> <span class="c1">#
{&#34;bank&#34;: ..., &#34;purchase_amount&#34;: ...}</span>
+<span class="n">output_pc</span> <span class="o">=</span> <span
class="n">input_pc</span> <span class="o">|</span> <span
class="n">beam</span><span class="o">.</span><span
class="n">Map</span><span class="p">(</span><span
class="k">lambda</span> <span class="n">item</span><span
class="p">:</span> <span class="n">beam</span><span
class="o">.</span><span class="n">Row</span><span
class="p">(</span>< [...]
+<span class="n">purchase_amount</span><span
class="o">=</span><span class="nb">float</span><span
class="p">(</span><span class="n">item</span><span
class="p">[</span><span
class="s2">&#34;purchase_amount&#34;</span><span
class="p">]))</span></code></pre></div>
+</div>
<h3 id="using-schemas">6.6. Using Schema Transforms</h3>
<p>A schema on a <code>PCollection</code> enables a rich variety of
relational transforms. The fact that each record is composed of
named fields allows for simple and readable aggregations that reference fields
by name, similar to the aggregations in
diff --git
a/website/generated-content/documentation/programming-guide/index.html
b/website/generated-content/documentation/programming-guide/index.html
index c895810..ee7b7d9 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -400,7 +400,7 @@ serialize and cache the values in your pipeline. Your
method should meet the
following requirements:</p><p class=language-java><ul><li>You should not in
any way modify an element returned by
the <code>@Element</code> annotation or
<code>ProcessContext.sideInput()</code> (the incoming
elements from the input collection).</li><li>Once you output a value using
<code>OutputReceiver.output()</code> you should not modify
-that value in any way.</li></ul></p><p class=language-python><ul><li>You
should not in any way modify the <code>element</code> argument provided to the
+that value in any way.</li></ul></p><p class=language-py><ul><li>You should
not in any way modify the <code>element</code> argument provided to the
<code>process</code> method, or any side inputs.</li><li>Once you output a
value using <code>yield</code> or <code>return</code>, you should not modify
that value in any way.</li></ul></p><h5 id=lightweight-dofns>4.2.1.3.
Lightweight DoFns and other abstractions</h5><p>If your function is relatively
straightforward, you can simplify your use of
<code>ParDo</code> by providing a lightweight <code>DoFn</code> in-line, as
@@ -1360,7 +1360,7 @@ types across different programming-language APIs.</p><p>A
<code>PCollection</cod
Schema rows; Beam uses a special coder to encode schema types.</p><h3
id=schemas-for-pl-types>6.2. Schemas for programming language
types</h3><p>While schemas themselves are language independent, they are
designed to embed naturally into the programming languages
of the Beam SDK being used. This allows Beam users to continue using native
types while reaping the advantage of
having Beam understand their element schemas.</p><p class=language-java>In
Java you could use the following set of classes to represent the purchase
schema. Beam will automatically
-infer the correct schema based on the members of the class.</p><div
class=language-java><div class=highlight><pre class=chroma><code
class=language-java data-lang=java><span class=nd>@DefaultSchema</span><span
class=o>(</span><span class=n>JavaBeanSchema</span><span class=o>.</span><span
class=na>class</span><span class=o>)</span>
+infer the correct schema based on the members of the class.</p><p
class=language-py>In Python you can use the following set of classes to
represent the purchase schema. Beam will automatically infer the correct schema
based on the members of the class.</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java
data-lang=java><span class=nd>@DefaultSchema</span><span class=o>(</span><span
class=n>JavaBeanSchema</span><span class=o>.</span><span class=na>c [...]
<span class=kd>public</span> <span class=kd>class</span> <span
class=nc>Purchase</span> <span class=o>{</span>
<span class=kd>public</span> <span class=n>String</span> <span
class=nf>getUserId</span><span class=o>();</span> <span class=c1>// Returns
the id of the user who made the purchase.
</span><span class=c1></span> <span class=kd>public</span> <span
class=kt>long</span> <span class=nf>getItemId</span><span class=o>();</span>
<span class=c1>// Returns the identifier of the item that was purchased.
@@ -1399,23 +1399,41 @@ infer the correct schema based on the members of the
class.</p><div class=langua
<span class=kd>public</span> <span class=nf>Transaction</span><span
class=o>(</span><span class=n>String</span> <span class=n>bank</span><span
class=o>,</span> <span class=kt>double</span> <span
class=n>purchaseAmount</span><span class=o>)</span> <span class=o>{</span>
<span class=o>...</span>
<span class=o>}</span>
-<span class=o>}</span></code></pre></div></div><p>Using JavaBean classes as
above is one way to map a schema to Java classes. However multiple Java classes
might have
+<span class=o>}</span></code></pre></div></div><div class=language-py><div
class=highlight><pre class=chroma><code class=language-py data-lang=py><span
class=kn>import</span> <span class=nn>typing</span>
+
+<span class=k>class</span> <span class=nc>Purchase</span><span
class=p>(</span><span class=n>typing</span><span class=o>.</span><span
class=n>NamedTuple</span><span class=p>):</span>
+ <span class=n>user_id</span><span class=p>:</span> <span class=nb>str</span>
<span class=c1># The id of the user who made the purchase.</span>
+ <span class=n>item_id</span><span class=p>:</span> <span class=nb>int</span>
<span class=c1># The identifier of the item that was purchased.</span>
+ <span class=n>shipping_address</span><span class=p>:</span> <span
class=n>ShippingAddress</span> <span class=c1># The shipping address, a nested
type.</span>
+ <span class=n>cost_cents</span><span class=p>:</span> <span
class=nb>int</span> <span class=c1># The cost of the item</span>
+ <span class=n>transactions</span><span class=p>:</span> <span
class=n>typing</span><span class=o>.</span><span class=n>Sequence</span><span
class=p>[</span><span class=n>Transaction</span><span class=p>]</span> <span
class=c1># The transactions that paid for this purchase (a list, since the
purchase might be spread out over multiple credit cards).</span>
+
+<span class=k>class</span> <span class=nc>ShippingAddress</span><span
class=p>(</span><span class=n>typing</span><span class=o>.</span><span
class=n>NamedTuple</span><span class=p>):</span>
+ <span class=n>street_address</span><span class=p>:</span> <span
class=nb>str</span>
+ <span class=n>city</span><span class=p>:</span> <span class=nb>str</span>
+ <span class=n>state</span><span class=p>:</span> <span
class=n>typing</span><span class=o>.</span><span class=n>Optional</span><span
class=p>[</span><span class=nb>str</span><span class=p>]</span>
+ <span class=n>country</span><span class=p>:</span> <span class=nb>str</span>
+ <span class=n>postal_code</span><span class=p>:</span> <span
class=nb>str</span>
+
+<span class=k>class</span> <span class=nc>Transaction</span><span
class=p>(</span><span class=n>typing</span><span class=o>.</span><span
class=n>NamedTuple</span><span class=p>):</span>
+ <span class=n>bank</span><span class=p>:</span> <span class=nb>str</span>
+ <span class=n>purchase_amount</span><span class=p>:</span> <span
class=nb>float</span></code></pre></div></div><p class=language-java>Using
JavaBean classes as above is one way to map a schema to Java classes. However
multiple Java classes might have
the same schema, in which case the different Java types can often be used
interchangeably. Beam will add implicit
conversions between types that have matching schemas. For example, the above
<code>Transaction</code> class has the same schema as the following
class:</p><div class=language-java><div class=highlight><pre class=chroma><code
class=language-java data-lang=java><span class=nd>@DefaultSchema</span><span
class=o>(</span><span class=n>JavaFieldSchema</span><span class=o>.</span><span
class=na>class</span><span class=o>)</span>
<span class=kd>public</span> <span class=kd>class</span> <span
class=nc>TransactionPojo</span> <span class=o>{</span>
<span class=kd>public</span> <span class=n>String</span> <span
class=n>bank</span><span class=o>;</span>
<span class=kd>public</span> <span class=kt>double</span> <span
class=n>purchaseAmount</span><span class=o>;</span>
-<span class=o>}</span></code></pre></div></div><p>So if we had two
<code>PCollection</code>s as follows</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java
data-lang=java><span class=n>PCollection</span><span class=o><</span><span
class=n>Transaction</span><span class=o>></span> <span
class=n>transactionBeans</span> <span class=o>=</span> <span
class=n>readTransactionsAsJavaBean</span><span class=o>();</span>
-<span class=n>PCollection</span><span class=o><</span><span
class=n>TransactionPojos</span><span class=o>></span> <span
class=n>transactionPojos</span> <span class=o>=</span> <span
class=n>readTransactionsAsPojo</span><span
class=o>();</span></code></pre></div></div><p>Then these two
<code>PCollection</code>s would have the same schema, even though their Java
types would be different. This means
+<span class=o>}</span></code></pre></div></div><p class=language-java>So if we
had two <code>PCollection</code>s as follows</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java
data-lang=java><span class=n>PCollection</span><span class=o><</span><span
class=n>Transaction</span><span class=o>></span> <span
class=n>transactionBeans</span> <span class=o>=</span> <span
class=n>readTransactionsAsJavaBean</span><span class=o>();</span>
+<span class=n>PCollection</span><span class=o><</span><span
class=n>TransactionPojos</span><span class=o>></span> <span
class=n>transactionPojos</span> <span class=o>=</span> <span
class=n>readTransactionsAsPojo</span><span
class=o>();</span></code></pre></div></div><p class=language-java>Then these
two <code>PCollection</code>s would have the same schema, even though their
Java types would be different. This means
for example the following two code snippets are valid:</p><div
class=language-java><div class=highlight><pre class=chroma><code
class=language-java data-lang=java><span class=n>transactionBeans</span><span
class=o>.</span><span class=na>apply</span><span class=o>(</span><span
class=n>ParDo</span><span class=o>.</span><span class=na>of</span><span
class=o>(</span><span class=k>new</span> <span class=n>DoFn</span><span
class=o><...>()</span> <span class=o>{</span>
<span class=nd>@ProcessElement</span> <span class=kd>public</span> <span
class=kt>void</span> <span class=nf>process</span><span class=o>(</span><span
class=nd>@Element</span> <span class=n>TransactionPojo</span> <span
class=n>pojo</span><span class=o>)</span> <span class=o>{</span>
<span class=o>...</span>
<span class=o>}</span>
-<span class=o>}));</span></code></pre></div></div><p>and<div
class=language-java><div class=highlight><pre class=chroma><code
class=language-java data-lang=java><span class=n>transactionPojos</span><span
class=o>.</span><span class=na>apply</span><span class=o>(</span><span
class=n>ParDo</span><span class=o>.</span><span class=na>of</span><span
class=o>(</span><span class=k>new</span> <span class=n>DoFn</span><span
class=o><...>()</span> <span class=o>{</span>
+<span class=o>}));</span></code></pre></div></div><p
class=language-java>and</p><div class=language-java><div class=highlight><pre
class=chroma><code class=language-java data-lang=java><span
class=n>transactionPojos</span><span class=o>.</span><span
class=na>apply</span><span class=o>(</span><span class=n>ParDo</span><span
class=o>.</span><span class=na>of</span><span class=o>(</span><span
class=k>new</span> <span class=n>DoFn</span><span class=o><...>()</span>
<span class=o>{</span>
<span class=nd>@ProcessElement</span> <span class=kd>public</span> <span
class=kt>void</span> <span class=nf>process</span><span class=o>(</span><span
class=nd>@Element</span> <span class=n>Transaction</span> <span
class=n>row</span><span class=o>)</span> <span class=o>{</span>
<span class=o>}</span>
-<span class=o>}));</span></code></pre></div></div></p><p>Even though the in
both cases the <code>@Element</code> parameter differs from the the
<code>PCollection</code>'s Java type, since the
+<span class=o>}));</span></code></pre></div></div><p class=language-java>Even
though the in both cases the <code>@Element</code> parameter differs from the
the <code>PCollection</code>'s Java type, since the
schemas are the same Beam will automatically make the conversion. The built-in
<code>Convert</code> transform can also be used
to translate between Java types of equivalent schemas, as detailed
below.</p><h3 id=schema-definition>6.3. Schema definition</h3><p>The schema for
a <code>PCollection</code> defines elements of that <code>PCollection</code> as
an ordered list of named fields. Each field
has a name, a type, and possibly a set of user options. The type of a field
can be primitive or composite. The following
@@ -1488,10 +1506,10 @@ getting just that field:</p><div
class=language-java><div class=highlight><pre c
<span class=k>case</span> <span class=s>"bytesField"</span><span
class=o>:</span>
<span class=k>return</span> <span class=n>processBytes</span><span
class=o>(</span><span class=n>oneOfValue</span><span class=o>.</span><span
class=na>getValue</span><span class=o>(</span><span class=n>bytes</span><span
class=o>[].</span><span class=na>class</span><span class=o>));</span>
<span class=o>}</span></code></pre></div></div><p>In the above example we used
the field names in the switch statement for clarity, however the enum integer
values could
-also be used.</p><h3 id=creating-schemas>6.5. Creating Schemas</h3><p>In order
to take advantage of schemas, your <code>PCollection</code>s must have a schema
attached to it. Often, the source itself will attach a schema to the
PCollection. For example, when using <code>AvroIO</code> to read Avro files,
the source can automatically infer a Beam schema from the Avro schema and
attach that to the Beam <code>PCollection</code>. However not all sources
produce schemas. In addition, often Bea [...]
+also be used.</p><h3 id=creating-schemas>6.5. Creating Schemas</h3><p>In order
to take advantage of schemas, your <code>PCollection</code>s must have a schema
attached to it. Often, the source itself will attach a schema to the
PCollection. For example, when using <code>AvroIO</code> to read Avro files,
the source can automatically infer a Beam schema from the Avro schema and
attach that to the Beam <code>PCollection</code>. However not all sources
produce schemas. In addition, often Bea [...]
Specification. A POJO can contain member variables that are primitives, that
are other POJOs, or are collections maps or
-arrays thereof. POJOs do not have to extend prespecified classes or extend any
specific interfaces.</p><p>If a POJO class is annotated with
<code>@DefaultSchema(JavaFieldSchema.class)</code>, Beam will automatically
infer a schema for
-this class. Nested classes are supported as are classes with
<code>List</code>, array, and <code>Map</code> fields.</p><p>For example,
annotating the following class tells Beam to infer a schema from this POJO
class and apply it to any
+arrays thereof. POJOs do not have to extend prespecified classes or extend any
specific interfaces.</p><p class=language-java>If a POJO class is annotated
with <code>@DefaultSchema(JavaFieldSchema.class)</code>, Beam will
automatically infer a schema for
+this class. Nested classes are supported as are classes with
<code>List</code>, array, and <code>Map</code> fields.</p><p
class=language-java>For example, annotating the following class tells Beam to
infer a schema from this POJO class and apply it to any
<code>PCollection<TransactionPojo></code>.</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java
data-lang=java><span class=nd>@DefaultSchema</span><span class=o>(</span><span
class=n>JavaFieldSchema</span><span class=o>.</span><span
class=na>class</span><span class=o>)</span>
<span class=kd>public</span> <span class=kd>class</span> <span
class=nc>TransactionPojo</span> <span class=o>{</span>
<span class=kd>public</span> <span class=kd>final</span> <span
class=n>String</span> <span class=n>bank</span><span class=o>;</span>
@@ -1503,17 +1521,17 @@ this class. Nested classes are supported as are classes
with <code>List</code>,
<span class=o>}</span>
<span class=o>}</span>
<span class=c1>// Beam will automatically infer the correct schema for this
PCollection. No coder is needed as a result.
-</span><span class=c1></span><span class=n>PCollection</span><span
class=o><</span><span class=n>TransactionPojo</span><span
class=o>></span> <span class=n>pojos</span> <span class=o>=</span> <span
class=n>readPojos</span><span class=o>();</span></code></pre></div></div><p>The
<code>@SchemaCreate</code> annotation tells Beam that this constructor can be
used to create instances of TransactionPojo,
+</span><span class=c1></span><span class=n>PCollection</span><span
class=o><</span><span class=n>TransactionPojo</span><span
class=o>></span> <span class=n>pojos</span> <span class=o>=</span> <span
class=n>readPojos</span><span class=o>();</span></code></pre></div></div><p
class=language-java>The <code>@SchemaCreate</code> annotation tells Beam that
this constructor can be used to create instances of TransactionPojo,
assuming that constructor parameters have the same names as the field names.
<code>@SchemaCreate</code> can also be used to annotate
static factory methods on the class, allowing the constructor to remain
private. If there is no <code>@SchemaCreate</code>
-annotation then all the fields must be non-final and the class must have a
zero-argument constructor.</p><p>There are a couple of other useful annotations
that affect how Beam infers schemas. By default the schema field names
+annotation then all the fields must be non-final and the class must have a
zero-argument constructor.</p><p class=language-java>There are a couple of
other useful annotations that affect how Beam infers schemas. By default the
schema field names
inferred will match that of the class field names. However
<code>@SchemaFieldName</code> can be used to specify a different name to
be used for the schema field. <code>@SchemaIgnore</code> can be used to mark
specific class fields as excluded from the inferred
schema. For example, it’s common to have ephemeral fields in a class that
should not be included in a schema
(e.g. caching the hash value to prevent expensive recomputation of the hash),
and <code>@SchemaIgnore</code> can be used to
-exclude these fields. Note that ignored fields will not be included in the
encoding of these records.</p><p>In some cases it is not convenient to annotate
the POJO class, for example if the POJO is in a different package that is
+exclude these fields. Note that ignored fields will not be included in the
encoding of these records.</p><p class=language-java>In some cases it is not
convenient to annotate the POJO class, for example if the POJO is in a
different package that is
not owned by the Beam pipeline author. In these cases the schema inference can
be triggered programmatically in
-pipeline’s main function as follows:</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java data-lang=java>
<span class=n>pipeline</span><span class=o>.</span><span
class=na>getSchemaRegistry</span><span class=o>().</span><span
class=na>registerPOJO</span><span class=o>(</span><span
class=n>TransactionPOJO</span><span class=o>.</span><span
class=na>class</span><span class=o>);</span></code></pre></div></div><h5
id=java-beans><strong>Java Beans</stron [...]
+pipeline’s main function as follows:</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java data-lang=java>
<span class=n>pipeline</span><span class=o>.</span><span
class=na>getSchemaRegistry</span><span class=o>().</span><span
class=na>registerPOJO</span><span class=o>(</span><span
class=n>TransactionPOJO</span><span class=o>.</span><span
class=na>class</span><span class=o>);</span></code></pre></div></div><p
class=language-java><strong>Java Beans</ [...]
standard has many characteristics, the key ones are that all properties are
accessed via getter and setter classes, and
the name format for these getters and setters is standardized. A Java Bean
class can be annotated with
<code>@DefaultSchema(JavaBeanSchema.class)</code> and Beam will automatically
infer a schema for this class. For example:</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java
data-lang=java><span class=nd>@DefaultSchema</span><span class=o>(</span><span
class=n>JavaBeanSchema</span><span class=o>.</span><span
class=na>class</span><span class=o>)</span>
@@ -1525,22 +1543,38 @@ the name format for these getters and setters is
standardized. A Java Bean class
<span class=kd>public</span> <span class=kt>void</span> <span
class=nf>setPurchaseAmount</span><span class=o>(</span><span
class=kt>double</span> <span class=n>purchaseAmount</span><span
class=o>)</span> <span class=o>{</span> <span class=err>…</span> <span
class=o>}</span>
<span class=o>}</span>
<span class=c1>// Beam will automatically infer the correct schema for this
PCollection. No coder is needed as a result.
-</span><span class=c1></span><span class=n>PCollection</span><span
class=o><</span><span class=n>TransactionBean</span><span
class=o>></span> <span class=n>beans</span> <span class=o>=</span> <span
class=n>readBeans</span><span class=o>();</span></code></pre></div></div><p>The
<code>@SchemaCreate</code> annotation can be used to specify a constructor or a
static factory method, in which case the
+</span><span class=c1></span><span class=n>PCollection</span><span
class=o><</span><span class=n>TransactionBean</span><span
class=o>></span> <span class=n>beans</span> <span class=o>=</span> <span
class=n>readBeans</span><span class=o>();</span></code></pre></div></div><p
class=language-java>The <code>@SchemaCreate</code> annotation can be used to
specify a constructor or a static factory method, in which case the
setters and zero-argument constructor can be omitted.</p><div
class=language-java><div class=highlight><pre class=chroma><code
class=language-java data-lang=java><span class=nd>@DefaultSchema</span><span
class=o>(</span><span class=n>JavaBeanSchema</span><span class=o>.</span><span
class=na>class</span><span class=o>)</span>
<span class=kd>public</span> <span class=kd>class</span> <span
class=nc>TransactionBean</span> <span class=o>{</span>
<span class=nd>@SchemaCreate</span>
<span class=n>Public</span> <span class=nf>TransactionBean</span><span
class=o>(</span><span class=n>String</span> <span class=n>bank</span><span
class=o>,</span> <span class=kt>double</span> <span
class=n>purchaseAmount</span><span class=o>)</span> <span class=o>{</span>
<span class=err>…</span> <span class=o>}</span>
<span class=kd>public</span> <span class=n>String</span> <span
class=nf>getBank</span><span class=o>()</span> <span class=o>{</span> <span
class=err>…</span> <span class=o>}</span>
<span class=kd>public</span> <span class=kt>double</span> <span
class=nf>getPurchaseAmount</span><span class=o>()</span> <span class=o>{</span>
<span class=err>…</span> <span class=o>}</span>
-<span
class=o>}</span></code></pre></div></div><p><code>@SchemaFieldName</code> and
<code>@SchemaIgnore</code> can be used to alter the schema inferred, just like
with POJO classes.</p><h5 id=autovalue><strong>AutoValue</strong></h5><p>Java
value classes are notoriously difficult to generate correctly. There is a lot
of boilerplate you must create in
+<span class=o>}</span></code></pre></div></div><p
class=language-java><code>@SchemaFieldName</code> and
<code>@SchemaIgnore</code> can be used to alter the schema inferred, just like
with POJO classes.</p><p class=language-java><strong>AutoValue</strong></p><p
class=language-java>Java value classes are notoriously difficult to generate
correctly. There is a lot of boilerplate you must create in
order to properly implement a value class. AutoValue is a popular library for
easily generating such classes by
-implementing a simple abstract base class.</p><p>Beam can infer a schema from
an AutoValue class. For example:</p><div class=language-java><div
class=highlight><pre class=chroma><code class=language-java
data-lang=java><span class=nd>@DefaultSchema</span><span class=o>(</span><span
class=n>AutoValueSchema</span><span class=o>.</span><span
class=na>class</span><span class=o>)</span>
+implementing a simple abstract base class.</p><p class=language-java>Beam can
infer a schema from an AutoValue class. For example:</p><div
class=language-java><div class=highlight><pre class=chroma><code
class=language-java data-lang=java><span class=nd>@DefaultSchema</span><span
class=o>(</span><span class=n>AutoValueSchema</span><span class=o>.</span><span
class=na>class</span><span class=o>)</span>
<span class=nd>@AutoValue</span>
<span class=kd>public</span> <span class=kd>abstract</span> <span
class=kd>class</span> <span class=nc>TransactionValue</span> <span
class=o>{</span>
<span class=kd>public</span> <span class=kd>abstract</span> <span
class=n>String</span> <span class=nf>getBank</span><span class=o>();</span>
<span class=kd>public</span> <span class=kd>abstract</span> <span
class=kt>double</span> <span class=nf>getPurchaseAmount</span><span
class=o>();</span>
-<span class=o>}</span></code></pre></div></div><p>This is all that’s needed to
generate a simple AutoValue class, and the above <code>@DefaultSchema</code>
annotation tells Beam to
-infer a schema from it. This also allows AutoValue elements to be used inside
of <code>PCollection</code>s.</p><p><code>@SchemaFieldName</code> and
<code>@SchemaIgnore</code> can be used to alter the schema inferred.</p><h3
id=using-schemas>6.6. Using Schema Transforms</h3><p>A schema on a
<code>PCollection</code> enables a rich variety of relational transforms. The
fact that each record is composed of
+<span class=o>}</span></code></pre></div></div><p class=language-java>This is
all that’s needed to generate a simple AutoValue class, and the above
<code>@DefaultSchema</code> annotation tells Beam to
+infer a schema from it. This also allows AutoValue elements to be used inside
of <code>PCollection</code>s.</p><p
class=language-java><code>@SchemaFieldName</code> and
<code>@SchemaIgnore</code> can be used to alter the schema inferred.</p><p
class=language-py>Beam has a few different mechanisms for inferring schemas
from Python code.</p><p class=language-py><strong>NamedTuple
classes</strong></p><p class=language-py>A <a
href=https://docs.python.org/3/library/typing.html#typing.NamedTup [...]
+class is a Python class that wraps a <code>tuple</code>, assigning a name to
each element
+and restricting it to a particular type. Beam will automatically infer the
+schema for PCollections with <code>NamedTuple</code> output types. For
example:</p><div class=language-py><div class=highlight><pre class=chroma><code
class=language-py data-lang=py><span class=k>class</span> <span
class=nc>Transaction</span><span class=p>(</span><span
class=n>typing</span><span class=o>.</span><span class=n>NamedTuple</span><span
class=p>):</span>
+ <span class=n>bank</span><span class=p>:</span> <span class=nb>str</span>
+ <span class=n>purchase_amount</span><span class=p>:</span> <span
class=nb>float</span>
+
+<span class=n>pc</span> <span class=o>=</span> <span class=nb>input</span>
<span class=o>|</span> <span class=n>beam</span><span class=o>.</span><span
class=n>Map</span><span class=p>(</span><span class=k>lambda</span> <span
class=o>...</span><span class=p>)</span><span class=o>.</span><span
class=n>with_output_types</span><span class=p>(</span><span
class=n>Transaction</span><span class=p>)</span></code></pre></div></div><p
class=language-py><strong>beam.Row</strong></p><p class=languag [...]
+that returns instances of <code>beam.Row</code>:</p><div
class=language-py><div class=highlight><pre class=chroma><code
class=language-py data-lang=py><span class=n>input_pc</span> <span
class=o>=</span> <span class=o>...</span> <span class=c1># {"bank":
..., "purchase_amount": ...}</span>
+<span class=n>output_pc</span> <span class=o>=</span> <span
class=n>input_pc</span> <span class=o>|</span> <span class=n>beam</span><span
class=o>.</span><span class=n>Map</span><span class=p>(</span><span
class=k>lambda</span> <span class=n>item</span><span class=p>:</span> <span
class=n>beam</span><span class=o>.</span><span class=n>Row</span><span
class=p>(</span><span class=n>bank</span><span class=o>=</span><span
class=n>item</span><span class=p>[</span><span class=s2>"bank" [...]
+ <span
class=n>purchase_amount</span><span class=o>=</span><span
class=n>item</span><span class=p>[</span><span
class=s2>"purchase_amount"</span><span
class=p>])</span></code></pre></div></div><p class=language-py>Note that this
declaration doesn’t include any specific information about the
+types of the <code>bank</code> and <code>purchase_amount</code> fields. Beam
will attempt to infer
+type information, if it’s unable to it will fall back to the generic type
+<code>Any</code>. Sometimes this is not ideal, you can use casts to make sure
Beam
+correctly infers types with <code>beam.Row</code>:</p><div
class=language-py><div class=highlight><pre class=chroma><code
class=language-py data-lang=py><span class=n>input_pc</span> <span
class=o>=</span> <span class=o>...</span> <span class=c1># {"bank":
..., "purchase_amount": ...}</span>
+<span class=n>output_pc</span> <span class=o>=</span> <span
class=n>input_pc</span> <span class=o>|</span> <span class=n>beam</span><span
class=o>.</span><span class=n>Map</span><span class=p>(</span><span
class=k>lambda</span> <span class=n>item</span><span class=p>:</span> <span
class=n>beam</span><span class=o>.</span><span class=n>Row</span><span
class=p>(</span><span class=n>bank</span><span class=o>=</span><span
class=nb>str</span><span class=p>(</span><span class=n>item</span><spa [...]
+ <span
class=n>purchase_amount</span><span class=o>=</span><span
class=nb>float</span><span class=p>(</span><span class=n>item</span><span
class=p>[</span><span class=s2>"purchase_amount"</span><span
class=p>]))</span></code></pre></div></div><h3 id=using-schemas>6.6. Using
Schema Transforms</h3><p>A schema on a <code>PCollection</code> enables a rich
variety of relational transforms. The fact that each record is composed of
named fields allows for simple and readable aggregations that reference fields
by name, similar to the aggregations in
a SQL expression.</p><h4 id=661-field-selection-syntax>6.6.1. Field selection
syntax</h4><p>The advantage of schemas is that they allow referencing of
element fields by name. Beam provides a selection syntax for
referencing fields, including nested and repeated fields. This syntax is used
by all of the schema transforms when
diff --git a/website/generated-content/sitemap.xml
b/website/generated-content/sitemap.xml
index 16088ee..37bba15 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.24.0/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/p
[...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.24.0/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/p
[...]
\ No newline at end of file
