This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d33bfc61441 Publishing website 2023/10/13 00:32:05 at commit 0586161
d33bfc61441 is described below
commit d33bfc61441dd20de294e8f18bac0bb545b8940d
Author: runner <runner@main-runner-7jxcw-j8rtj>
AuthorDate: Fri Oct 13 00:32:05 2023 +0000
Publishing website 2023/10/13 00:32:05 at commit 0586161
---
website/generated-content/documentation/index.xml | 2 +-
.../documentation/ml/multi-language-inference/index.html | 2 +-
.../documentation/sdks/python-pipeline-dependencies/index.html | 9 +++++----
website/generated-content/sitemap.xml | 2 +-
4 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/website/generated-content/documentation/index.xml
b/website/generated-content/documentation/index.xml
index 04a4369e3d2..43288b91049 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -17783,7 +17783,7 @@ used to predict the last word of a sentence based on
the context of the sentence
<p>Finally, we postprocess the model predictions in the
<code>Postprocess</code> DoFn. The <code>Postprocess</code> DoFn
returns the original text, the last word of the sentence, and the predicted
word.</p>
<h3 id="compile-python-code-into-package">Compile Python code into
package</h3>
<p>The custom Python code needs to be written in a local package and be
compiled as a tarball. This package can then be used by the Java pipeline. The
following example shows how to compile the Python package into a tarball:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code
class="language-bash" data-lang="bash"><span class="line"><span
class="cl"> python setup.py sdist
+<div class="highlight"><pre tabindex="0" class="chroma"><code
class="language-bash" data-lang="bash"><span class="line"><span
class="cl"> pip install --upgrade build <span
class="o">&amp;&amp;</span> python -m build --sdist
</span></span></code></pre></div><p>In order to run this, a
<code>setup.py</code> is required. The path to the tarball will be used
as an argument in the pipeline options of the Java pipeline.</p>
<h3 id="run-the-java-pipeline">Run the Java pipeline</h3>
<p>The Java pipeline is defined in the <a
href="https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/apache/beam/examples/MultiLangRunInference.java#L32";><code>MultiLangRunInference</code></a>
class. In this pipeline, the data is read from Google Cloud Storage, the
cross-language Python transform is applied, and the output is written back to
Google Cloud Storage.</p>
diff --git
a/website/generated-content/documentation/ml/multi-language-inference/index.html
b/website/generated-content/documentation/ml/multi-language-inference/index.html
index 9430730ff8b..f20448e6cff 100644
---
a/website/generated-content/documentation/ml/multi-language-inference/index.html
+++
b/website/generated-content/documentation/ml/multi-language-inference/index.html
@@ -61,7 +61,7 @@ used to predict the last word of a sentence based on the
context of the sentence
</span></span><span class=line><span class=cl>
</span></span><span class=line><span class=cl> <span class=k>def</span>
<span class=nf>batch_elements_kwargs</span><span class=p>(</span><span
class=bp>self</span><span class=p>):</span>
</span></span><span class=line><span class=cl> <span
class=k>return</span> <span class=p>{</span><span
class=s1>'max_batch_size'</span><span class=p>:</span> <span
class=mi>1</span><span class=p>}</span>
-</span></span></code></pre></div><p>An alternative aproach is to make all the
tensors have the same length. This <a
href=https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb>example</a>
shows how to do that.</p><p>The <code>ModelConfig</code> and
<code>ModelTokenizer</code> are loaded in the initialization function. The
<code>ModelConfig</code> is used to define the model architecture, and the
<code>ModelTokenizer</code> is [...]
+</span></span></code></pre></div><p>An alternative aproach is to make all the
tensors have the same length. This <a
href=https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb>example</a>
shows how to do that.</p><p>The <code>ModelConfig</code> and
<code>ModelTokenizer</code> are loaded in the initialization function. The
<code>ModelConfig</code> is used to define the model architecture, and the
<code>ModelTokenizer</code> is [...]
</span></span></code></pre></div><p>In order to run this, a
<code>setup.py</code> is required. The path to the tarball will be used as an
argument in the pipeline options of the Java pipeline.</p><h3
id=run-the-java-pipeline>Run the Java pipeline</h3><p>The Java pipeline is
defined in the <a
href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/apache/beam/examples/MultiLangRunInference.j
[...]
</span></span></span><span class=line><span class=cl><span class=se></span>
-Dexec.args<span class=o>=</span><span class=s2>"--runner=DataflowRunner \
</span></span></span><span class=line><span class=cl><span class=s2>
--project=</span><span class=nv>$GCP_PROJECT</span><span class=s2>\
diff --git
a/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
b/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
index d7aa54de2fd..9bbf61ed886 100644
---
a/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
+++
b/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
@@ -41,10 +41,11 @@
</code></pre><p>The runner will use the <code>requirements.txt</code> file to
install your additional dependencies onto the remote
workers.</p></li></ol><blockquote><p><strong>NOTE</strong>: An alternative to
<code>pip freeze</code> is to use a library like <a
href=https://github.com/jazzband/pip-tools>pip-tools</a> to compile all the
dependencies required for the pipeline from a <code>--requirements_file</code>,
where only top-level dependencies are mentioned.</p></blockquote><h2 id=cus
[...]
COPY <path to requirements.txt> /tmp/requirements.txt
RUN python -m pip install -r /tmp/requirements.txt
-</code></pre></li></ol><h2 id=local-or-nonpypi>Local or non-PyPI
Dependencies</h2><p>If your pipeline uses packages that are not available
publicly (e.g. packages that you’ve downloaded from a GitHub repo), make
these packages available remotely by performing the following
steps:</p><ol><li><p>Identify which packages are installed on your machine and
are not public. Run the following command:</p><p>pip freeze</p><p>This command
lists all packages that are installed on your machine, [...]
-</code></pre><p>where package-name is the package’s tarball. If you have
the <code>setup.py</code> for that
-package then you can build the tarball with the following
command:</p><pre><code> python setup.py sdist
-</code></pre><p>See the <a
href=https://docs.python.org/3/distutils/sourcedist.html>sdist
documentation</a> for more details on this command.</p></li></ol><h2
id=multiple-file-dependencies>Multiple File Dependencies</h2><p>Often, your
pipeline code spans multiple files. To run your project remotely, you must
group these files as a Python package and specify the package when you run your
pipeline. When the remote workers start, they will install your package. To
group your files as a Pyth [...]
+</code></pre></li></ol><h2 id=local-or-nonpypi>Local or non-PyPI
Dependencies</h2><p>If your pipeline uses packages that are not available
publicly (e.g. packages that you’ve downloaded from a GitHub repo), make
these packages available remotely by performing the following
steps:</p><ol><li><p>Identify which packages are installed on your machine and
are not public. Run the following command:</p><p>pip freeze</p><p>This command
lists all packages that are installed on your machine, [...]
+</code></pre><p>where package-name is the package’s tarball. You can
build the package tarball using a command line tool called <a
href=https://setuptools.pypa.io/en/latest/userguide/quickstart.html#install-build>build</a>.</p><pre><code>
# Install build using pip
+ pip install --upgrade build
+ python -m build --sdist
+</code></pre><p>See the <a
href=https://pypa-build.readthedocs.io/en/latest/index.html>build
documentation</a> for more details on this command.</p></li></ol></li></ol><h2
id=multiple-file-dependencies>Multiple File Dependencies</h2><p>Often, your
pipeline code spans multiple files. To run your project remotely, you must
group these files as a Python package and specify the package when you run your
pipeline. When the remote workers start, they will install your package. To
group your fi [...]
setuptools.setup(
name='PACKAGE-NAME',
diff --git a/website/generated-content/sitemap.xml
b/website/generated-content/sitemap.xml
index 586f2a808e9..c9a70ac9180 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.51.0/</loc><lastmod>2023-10-12T18:57:35+04:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2023-10-12T18:57:35+04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2023-10-12T18:57:35+04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2023-10-12T18:57:35+04:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.51.0/</loc><lastmod>2023-10-12T13:20:39-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2023-10-12T13:20:39-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2023-10-12T13:20:39-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2023-10-12T13:20:39-07:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
