This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new dde043d4d78 Publishing website 2024/03/11 23:37:27 at commit d22a7e7
dde043d4d78 is described below
commit dde043d4d78f4affaf5329450e3a96f92f60c58e
Author: runner <runner@main-runner-zt478-tf6m6>
AuthorDate: Mon Mar 11 23:37:28 2024 +0000
Publishing website 2024/03/11 23:37:27 at commit d22a7e7
---
website/generated-content/documentation/index.xml | 13 ++++++-------
.../documentation/programming-guide/index.html | 7 +++++--
website/generated-content/sitemap.xml | 2 +-
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/website/generated-content/documentation/index.xml
b/website/generated-content/documentation/index.xml
index 432429ce373..d48748176b7 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -15484,10 +15484,11 @@ $ jar -jar /path/to/expansion_service.jar
&lt;PORT_NUMBER&gt;</code>&
</span></span><span class="line"><span class="cl"> <span
class="n">PythonExternalTransform</span><span
class="o">.&lt;</span><span class="n">PBegin</span><span
class="o">,</span> <span class="n">PCollection</span><span
class="o">&lt;</span><span class="n">String</span><span
class="o">&gt;&gt;</span><span class="n">from</span><span
class="o">(</span><span class="s">&#34;apache_beam.io.ReadFromTe [...]
</span></span><span class="line"><span class="cl"> <span
class="o">.</span><span class="na">withKwarg</span><span
class="o">(</span><span
class="s">&#34;file_pattern&#34;</span><span
class="o">,</span> <span class="n">options</span><span
class="o">.</span><span class="na">getInputFile</span><span
class="o">())</span>
</span></span><span class="line"><span class="cl"> <span
class="o">.</span><span class="na">withKwarg</span><span
class="o">(</span><span
class="s">&#34;validate&#34;</span><span class="o">,</span>
<span class="kc">false</span><span class="o">))</span>
-</span></span></code></pre></div><blockquote>
-<p><strong>Note:</strong> <code>PythonExternalTransform</code>
has other useful methods such as <code>withExtraPackages</code> for
staging PyPI package dependencies and <code>withOutputCoder</code> for
setting an output coder.</p>
-</blockquote>
-<p>Alternatively, you may want to create a Python module that registers an
existing Python transform as a cross-language transform for use with the Python
expansion service and calls into that existing transform to perform its
intended operation. A registered URN can be used later in an expansion request
for indicating an expansion target.</p>
+</span></span></code></pre></div><p><code>PythonExternalTransform</code>
has other useful methods such as <code>withExtraPackages</code> for
staging PyPI package dependencies and <code>withOutputCoder</code> for
setting an output coder. If your transform exists in an external package, make
sure to specify that package using <code>withExtraPackages</code>, for
example:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code
class="language-java" data-lang="java"><span class="line"><span
class="cl"><span class="n">p</span><span class="o">.</span><span
class="na">apply</span><span class="o">(</span><span
class="s">&#34;Read&#34;</span><span class="o">,</span>
+</span></span><span class="line"><span class="cl"> <span
class="n">PythonExternalTransform</span><span
class="o">.&lt;</span><span class="n">PBegin</span><span
class="o">,</span> <span class="n">PCollection</span><span
class="o">&lt;</span><span class="n">String</span><span
class="o">&gt;&gt;</span><span class="n">from</span><span
class="o">(</span><span class="s">&#34;my_python_package.BeamRea [...]
+</span></span><span class="line"><span class="cl"> <span
class="o">.</span><span class="na">withExtraPackages</span><span
class="o">(</span><span class="n">ImmutableList</span><span
class="o">.</span><span class="na">of</span><span
class="o">(</span><span
class="s">&#34;my_python_package&#34;</span><span
class="o">)))</span>
+</span></span></code></pre></div><p>Alternatively, you may
want to create a Python module that registers an existing Python transform as a
cross-language transform for use with the Python expansion service and calls
into that existing transform to perform its intended operation. A registered
URN can be used later in an expansion request for indicating an expansion
target.</p>
<p><strong>Defining the Python module</strong></p>
<ol>
<li>
@@ -15551,11 +15552,9 @@ $ jar -jar /path/to/expansion_service.jar
&lt;PORT_NUMBER&gt;</code>&
</div>
</li>
<li>
-<p>This expansion service is now ready to serve up transforms on the
address <code>localhost:$PORT_FOR_EXPANSION_SERVICE</code>.</p>
+<p>This expansion service is now ready to serve up transforms on the
address `localhost:$PORT_FOR_EXPANSION_SERVICE</p>
</li>
</ol>
-<p><strong>Including dependencies</strong></p>
-<p>Currently Python external transforms are limited to dependencies
available in the core Beam SDK harness.</p>
<h4 id="1313-creating-cross-language-go-transforms">13.1.3. Creating
cross-language Go transforms</h4>
<p>Go currently does not support creating cross-language transforms, only
using cross-language
transforms from other languages; see more at <a
href="https://github.com/apache/beam/issues/21767";>Issue 21767</a>.</p>
diff --git
a/website/generated-content/documentation/programming-guide/index.html
b/website/generated-content/documentation/programming-guide/index.html
index 45ab1f54854..3f1c5feab3a 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -5595,7 +5595,10 @@ $ jar -jar /path/to/expansion_service.jar
<PORT_NUMBER></code></pre></div>
</span></span><span class=line><span class=cl> <span
class=n>PythonExternalTransform</span><span class=o>.<</span><span
class=n>PBegin</span><span class=o>,</span> <span
class=n>PCollection</span><span class=o><</span><span
class=n>String</span><span class=o>>></span><span
class=n>from</span><span class=o>(</span><span
class=s>"apache_beam.io.ReadFromText"</span><span class=o>)</span>
</span></span><span class=line><span class=cl> <span class=o>.</span><span
class=na>withKwarg</span><span class=o>(</span><span
class=s>"file_pattern"</span><span class=o>,</span> <span
class=n>options</span><span class=o>.</span><span
class=na>getInputFile</span><span class=o>())</span>
</span></span><span class=line><span class=cl> <span class=o>.</span><span
class=na>withKwarg</span><span class=o>(</span><span
class=s>"validate"</span><span class=o>,</span> <span
class=kc>false</span><span class=o>))</span>
-</span></span></code></pre></div><blockquote><p><strong>Note:</strong>
<code>PythonExternalTransform</code> has other useful methods such as
<code>withExtraPackages</code> for staging PyPI package dependencies and
<code>withOutputCoder</code> for setting an output
coder.</p></blockquote><p>Alternatively, you may want to create a Python module
that registers an existing Python transform as a cross-language transform for
use with the Python expansion service and calls into that existing tr [...]
+</span></span></code></pre></div><p><code>PythonExternalTransform</code> has
other useful methods such as <code>withExtraPackages</code> for staging PyPI
package dependencies and <code>withOutputCoder</code> for setting an output
coder. If your transform exists in an external package, make sure to specify
that package using <code>withExtraPackages</code>, for example:</p><div
class=highlight><pre tabindex=0 class=chroma><code class=language-java
data-lang=java><span class=line><span clas [...]
+</span></span><span class=line><span class=cl> <span
class=n>PythonExternalTransform</span><span class=o>.<</span><span
class=n>PBegin</span><span class=o>,</span> <span
class=n>PCollection</span><span class=o><</span><span
class=n>String</span><span class=o>>></span><span
class=n>from</span><span class=o>(</span><span
class=s>"my_python_package.BeamReadPTransform"</span><span
class=o>)</span>
+</span></span><span class=line><span class=cl> <span class=o>.</span><span
class=na>withExtraPackages</span><span class=o>(</span><span
class=n>ImmutableList</span><span class=o>.</span><span class=na>of</span><span
class=o>(</span><span class=s>"my_python_package"</span><span
class=o>)))</span>
+</span></span></code></pre></div><p>Alternatively, you may want to create a
Python module that registers an existing Python transform as a cross-language
transform for use with the Python expansion service and calls into that
existing transform to perform its intended operation. A registered URN can be
used later in an expansion request for indicating an expansion
target.</p><p><strong>Defining the Python module</strong></p><ol><li><p>Define
a Uniform Resource Name (URN) for your transfo [...]
</span></span></code></pre></div></li><li><p>For an existing Python transform,
create a new class to register the URN with the Python expansion
service.</p><div class=highlight><pre tabindex=0 class=chroma><code
class=language-py data-lang=py><span class=line><span class=cl><span
class=nd>@ptransform.PTransform.register_urn</span><span class=p>(</span><span
class=n>TEST_COMPK_URN</span><span class=p>,</span> <span
class=kc>None</span><span class=p>)</span>
</span></span><span class=line><span class=cl><span class=k>class</span> <span
class=nc>CombinePerKeyTransform</span><span class=p>(</span><span
class=n>ptransform</span><span class=o>.</span><span
class=n>PTransform</span><span class=p>):</span>
</span></span></code></pre></div></li><li><p>From within the class, define an
expand method that takes an input PCollection, runs the Python transform, and
then returns the output PCollection.</p><div class=highlight><pre tabindex=0
class=chroma><code class=language-py data-lang=py><span class=line><span
class=cl><span class=k>def</span> <span class=nf>expand</span><span
class=p>(</span><span class=bp>self</span><span class=p>,</span> <span
class=n>pcoll</span><span class=p>):</span>
@@ -5610,7 +5613,7 @@ $ jar -jar /path/to/expansion_service.jar
<PORT_NUMBER></code></pre></div>
</span></span><span class=line><span class=cl> <span class=k>return</span>
<span class=n>CombinePerKeyTransform</span><span class=p>()</span>
</span></span></code></pre></div></li></ol><p><strong>Starting the expansion
service</strong></p><p>An expansion service can be used with multiple
transforms in the same pipeline. The Beam Python SDK provides a default
expansion service for you to use with your Python transforms. You are free to
write your own expansion service, but that is generally not needed, so it is
not covered in this section.</p><p>Perform the following steps to start up the
default Python expansion service direct [...]
</code></pre></div></div></li><li><p>Import any modules that contain
transforms to be made available using the expansion service.</p><div
class=snippet><div class="notebook-skip code-snippet without_switcher"><a
class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom
title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre
tabindex=0><code>$ python -m
apache_beam.runners.portability.expansion_service_test -p
$PORT_FOR_EXPANSION_SERVICE --pickle_library=cloudpickle
- </code></pre></div></div></li><li><p>This expansion service is now ready
to serve up transforms on the address
<code>localhost:$PORT_FOR_EXPANSION_SERVICE</code>.</p></li></ol><p><strong>Including
dependencies</strong></p><p>Currently Python external transforms are limited
to dependencies available in the core Beam SDK harness.</p><h4
id=1313-creating-cross-language-go-transforms>13.1.3. Creating cross-language
Go transforms</h4><p>Go currently does not support creating cross-languag [...]
+ </code></pre></div></div></li><li><p>This expansion service is now ready
to serve up transforms on the address
`localhost:$PORT_FOR_EXPANSION_SERVICE</p></li></ol><h4
id=1313-creating-cross-language-go-transforms>13.1.3. Creating cross-language
Go transforms</h4><p>Go currently does not support creating cross-language
transforms, only using cross-language
transforms from other languages; see more at <a
href=https://github.com/apache/beam/issues/21767>Issue 21767</a>.</p><h4
id=1314-defining-a-urn>13.1.4. Defining a URN</h4><p>Developing a
cross-language transform involves defining a URN for registering the transform
with an expansion service. In this section
we provide a convention for defining such URNs. Following this convention is
optional but it will ensure that your transform
will not run into conflicts when registering in an expansion service along
with transforms developed by other developers.</p><h5 id=13141-schema>13.1.4.1.
Schema</h5><p>A URN should consist of the following
components:</p><ul><li><strong>ns-id</strong>: A namespace identifier. Default
recommendation is
<code>beam:transform</code>.</li><li><strong>org-identifier</strong>:
Identifies the organization where the transform was defined. Transforms defined
in Apache Beam use <code>org.apache.be [...]
diff --git a/website/generated-content/sitemap.xml
b/website/generated-content/sitemap.xml
index bf227246272..86bcc10879c 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.54.0/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.54.0/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
