This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new 7fbc8c9a02c Publishing website 2023/08/17 22:17:50 at commit c7b7921 7fbc8c9a02c is described below commit 7fbc8c9a02c4e62bc484f108291b0f7a073eaa72 Author: jenkins <bui...@apache.org> AuthorDate: Thu Aug 17 22:17:50 2023 +0000 Publishing website 2023/08/17 22:17:50 at commit c7b7921 --- website/generated-content/documentation/index.xml | 64 +++++++++++++--------- .../documentation/programming-guide/index.html | 19 +++++-- website/generated-content/sitemap.xml | 2 +- 3 files changed, 51 insertions(+), 34 deletions(-) diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml index b20f10d6cc4..563515ddbd4 100644 --- a/website/generated-content/documentation/index.xml +++ b/website/generated-content/documentation/index.xml @@ -15447,34 +15447,41 @@ implementations. This information can be accessed by using the conventional </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">input_type</span></span></span></code></pre></div> </div> </div> -<h2 id="transform-service">15 Transform Service</h2> -<p>Starting version 2.49.0, Beam introduced a <a href="https://docs.docker.com/compose/">Docker Compose</a> based service named <em>Transform Service</em>. The Transform Service allows Beam portable -pipelines to perform expansion of supported transforms using Docker.</p> -<p>The basic architecture of the Transform Service is given below.</p> -<p><img src="/images/transform_service.png" alt="Diagram of the transform service architecture"></p> -<p>The Transform Service can be useful in many contexts. We have identified two primary use-cases below. Note that to use the transform service, Docker (and Docker Compose) needs to be available in the machine where the service will be started at.</p> +<h2 id="transform-service">15 Transform service</h2> +<p>The Apache Beam SDK versions 2.49.0 and later include a <a href="https://docs.docker.com/compose/">Docker Compose</a> +service named <em>Transform service</em>. Use the Transform service to perform expansions of supported transforms +on Beam portable pipelines by using Docker.</p> +<p>The following diagram illustrates the basic architecture of the Transform service.</p> +<p><img src="/images/transform_service.png" alt="Diagram of the Transform service architecture"></p> +<p>To use the Transform service, Docker and Docker Compose must be available on the machine that starts the service.</p> +<p>The Transform service has the following primary use cases:</p> <ul> -<li>Perform expansion of cross-language transforms without installing other language runtimes.</li> -</ul> -<p>Transforms Service allows multi-language pipelines to use/expand cross-language transforms implemented in other SDKs without installing runtimes for implementation languages of such SDKs. -For example, with the Transform Service, a Beam Python pipeline can use Java GCP I/O transforms and Java Kafka I/O transforms without installing a Java runtime locally.</p> -<ul> -<li>Upgrade transforms without upgrading the Beam version.</li> +<li> +<p>Perform expansion of cross-language transforms without installing other language runtimes.</p> +<p>The Transform service allows multi-language pipelines to use and expand cross-language transforms implemented +in other SDKs without requiring you to install runtimes for the implementation languages of those SDKs. +For example, with the Transform service, a Beam Python pipeline can use the Google Cloud Java I/O transforms and Java Kafka I/O transforms +without a local Java runtime installation.</p> +</li> +<li> +<p>Upgrade transforms without upgrading the Apache Beam SDK version.</p> +<p>Use the Transform service to upgrade the Beam SDK versions of individual transforms used by Beam pipelines without upgrading the Beam version of the pipeline. +This feature is currently in development. For more details, see +<a href="https://github.com/apache/beam/issues/27943">GitHub issue #27943: Upgrade transforms without upgrading the pipeline using the Transform Service</a>.</p> +</li> </ul> -<p>The transform service can be used to upgrade individual transforms used by Beam pipelines to a new Beam version without upgrading the Beam version used by the pipeline. -This feature is currently in development. Please see the <a href="https://github.com/apache/beam/issues/27943">tracking issue</a> for more details.</p> -<h3 id="transform-service-usage">15.1 Using the the Transform Service</h3> -<p>Beam SDKs may automatically startup a transform service to perform expansion when using cross-language transforms. More specifically,</p> +<h3 id="transform-service-usage">15.1 Use the Transform service</h3> +<p>In some cases, Apache Beam SDKs automatically start the Transform service, such as in the following scenarios:</p> <ul> <li> -<p>Java <a href="https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/PythonExternalTransform.java">PythonExternalTransform API</a> will automatically -startup the Transform Service for you if a Python runtime is not available locally but Docker is available.</p> +<p>The Java <a href="https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/PythonExternalTransform.java"><code>PythonExternalTransform</code> API</a> automatically +starts the Transform service when a Python runtime isn&rsquo;t available locally, but Docker is.</p> </li> <li> -<p>Beam Python multi-language wrappers may automatically startup a Transform Service for you when using Java transforms, if a Java language runtime is not available locally but Docker is available.</p> +<p>The Apache Beam Python multi-language wrappers might automatically start the Transform service when you&rsquo;re using Java transforms, a Java language runtime isn&rsquo;t available locally, and Docker is available locally.</p> </li> </ul> -<p>Additionally, if needed, a Transform Service instance can be manually started using utilities provided with Beam SDKs.</p> +<p>To manually start a Transform service instance by using utilities provided with the Apache Beam SDKs, use the following commands.</p> <div class='language-java snippet'> <div class="notebook-skip code-snippet"> <a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard"> @@ -15499,7 +15506,7 @@ startup the Transform Service for you if a Python runtime is not available local <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="nx">This</span> <span class="nx">feature</span> <span class="nx">is</span> <span class="nx">currently</span> <span class="nx">in</span> <span class="nx">development</span><span class="p">.</span></span></span></code></pre></div> </div> </div> -<p>To stop the transform service use the following commands.</p> +<p>To stop the transform service, use the following commands.</p> <div class='language-java snippet'> <div class="notebook-skip code-snippet"> <a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard"> @@ -15524,18 +15531,21 @@ startup the Transform Service for you if a Python runtime is not available local <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="nx">This</span> <span class="nx">feature</span> <span class="nx">is</span> <span class="nx">currently</span> <span class="nx">in</span> <span class="nx">development</span><span class="p">.</span></span></span></code></pre></div> </div> </div> -<h3 id="transform-service-included-transforms">15.2 Portable Transforms included in the Transform Service</h3> -<p>Transforms service includes a number of portable transforms implemented in Beam Java and Python SDKs.</p> -<p>Some of the transforms currently included in the Trasnform Service are given below.</p> +<h3 id="transform-service-included-transforms">15.2 Portable transforms included in the Transform service</h3> +<p>The Transform service includes portable transforms implemented in the Apache Beam Java and Python SDKs.</p> +<p>The following transforms are included in the Trasnform service:</p> <ul> <li> -<p>Java transforms - GCP I/O connectors, Kafka I/O connector, JDBC I/O connector.</p> +<p>Java transforms: Google Cloud I/O connectors, the Kafka I/O connector, and the JDBC I/O connector</p> </li> <li> -<p>Python transforms - all portable transforms implemented within Beam Python SDK, for example, RunInference and Dataframe transforms.</p> +<p>Python transforms: all portable transforms implemented within the Apache Beam Python SDK, such as +<a href="/documentation/transforms/python/elementwise/runinference/">RunInference</a> and +<a href="/documentation/dsls/dataframes/overview/">DataFrame</a> transforms.</p> </li> </ul> -<p>For a more detailed list of available transforms, please see <a href="https://cwiki.apache.org/confluence/display/BEAM/Transform+Service">here</a>.</p></description></item><item><title>Documentation: BigQuery ML integration</title><link>/documentation/patterns/bqml/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/documentation/patterns/bqml/</guid><description> +<p>For a more comprehensive list of available transforms, see the +<a href="https://cwiki.apache.org/confluence/display/BEAM/Transform+Service">Transform service</a> developer guide.</p></description></item><item><title>Documentation: BigQuery ML integration</title><link>/documentation/patterns/bqml/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/documentation/patterns/bqml/</guid><description> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html index 90db53219c9..53bc461fab1 100644 --- a/website/generated-content/documentation/programming-guide/index.html +++ b/website/generated-content/documentation/programming-guide/index.html @@ -36,7 +36,7 @@ <img class=banner-img-mobile src=/images/banners/tour-of-beam/tour-of-beam-mobile.png alt="Start Tour of Beam"></a></div><div class=swiper-slide><a href=https://beam.apache.org/documentation/ml/overview/><img class=banner-img-desktop src=/images/banners/machine-learning/machine-learning-desktop.jpg alt="Machine Learning"> <img class=banner-img-mobile src=/images/banners/machine-learning/machine-learning-mobile.jpg alt="Machine Learning"></a></div></div><div class=swiper-pagination></div></div><script src=/js/swiper-bundle.min.min.e0e8f81b0b15728d35ff73c07f42ddbb17a108d6f23df4953cb3e60df7ade675.js></script> <script src=/js/sliders/top-banners.min.91104c476b3d8123ebee5ed9a8168556ec546abb698549551b38a0cee187ee1c.js></script> -<script>function showSearch(){addPlaceholder();var e,t=document.querySelector(".searchBar");t.classList.remove("disappear"),e=document.querySelector("#iconsBar"),e.classList.add("disappear")}function addPlaceholder(){$("input:text").attr("placeholder","What are you looking for?")}function endSearch(){var e,t=document.querySelector(".searchBar");t.classList.add("disappear"),e=document.querySelector("#iconsBar"),e.classList.remove("disappear")}function blockScroll(){$("body").toggleClass(" [...] +<script>function showSearch(){addPlaceholder();var e,t=document.querySelector(".searchBar");t.classList.remove("disappear"),e=document.querySelector("#iconsBar"),e.classList.add("disappear")}function addPlaceholder(){$("input:text").attr("placeholder","What are you looking for?")}function endSearch(){var e,t=document.querySelector(".searchBar");t.classList.add("disappear"),e=document.querySelector("#iconsBar"),e.classList.remove("disappear")}function blockScroll(){$("body").toggleClass(" [...] Beam SDKs to create data processing pipelines. It provides guidance for using the Beam SDK classes to build and test your pipeline. The programming guide is not intended as an exhaustive reference, but as a language-agnostic, high-level @@ -5633,11 +5633,18 @@ implementations. This information can be accessed by using the conventional </span></span><span class=line><span class=cl> <span class=o>...</span> </span></span><span class=line><span class=cl> </span></span><span class=line><span class=cl> <span class=k>def</span> <span class=nf>infer_output_type</span><span class=p>(</span><span class=bp>self</span><span class=p>,</span> <span class=n>input_type</span><span class=p>):</span> -</span></span><span class=line><span class=cl> <span class=k>return</span> <span class=n>input_type</span></span></span></code></pre></div></div></div><h2 id=transform-service>15 Transform Service</h2><p>Starting version 2.49.0, Beam introduced a <a href=https://docs.docker.com/compose/>Docker Compose</a> based service named <em>Transform Service</em>. The Transform Service allows Beam portable -pipelines to perform expansion of supported transforms using Docker.</p><p>The basic architecture of the Transform Service is given below.</p><p><img src=/images/transform_service.png alt="Diagram of the transform service architecture"></p><p>The Transform Service can be useful in many contexts. We have identified two primary use-cases below. Note that to use the transform service, Docker (and Docker Compose) needs to be available in the machine where the service will be started at.</p>< [...] -For example, with the Transform Service, a Beam Python pipeline can use Java GCP I/O transforms and Java Kafka I/O transforms without installing a Java runtime locally.</p><ul><li>Upgrade transforms without upgrading the Beam version.</li></ul><p>The transform service can be used to upgrade individual transforms used by Beam pipelines to a new Beam version without upgrading the Beam version used by the pipeline. -This feature is currently in development. Please see the <a href=https://github.com/apache/beam/issues/27943>tracking issue</a> for more details.</p><h3 id=transform-service-usage>15.1 Using the the Transform Service</h3><p>Beam SDKs may automatically startup a transform service to perform expansion when using cross-language transforms. More specifically,</p><ul><li><p>Java <a href=https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/ex [...] -startup the Transform Service for you if a Python runtime is not available locally but Docker is available.</p></li><li><p>Beam Python multi-language wrappers may automatically startup a Transform Service for you when using Java transforms, if a Java language runtime is not available locally but Docker is available.</p></li></ul><p>Additionally, if needed, a Transform Service instance can be manually started using utilities provided with Beam SDKs.</p><div class='language-java snippet'>< [...] +</span></span><span class=line><span class=cl> <span class=k>return</span> <span class=n>input_type</span></span></span></code></pre></div></div></div><h2 id=transform-service>15 Transform service</h2><p>The Apache Beam SDK versions 2.49.0 and later include a <a href=https://docs.docker.com/compose/>Docker Compose</a> +service named <em>Transform service</em>. Use the Transform service to perform expansions of supported transforms +on Beam portable pipelines by using Docker.</p><p>The following diagram illustrates the basic architecture of the Transform service.</p><p><img src=/images/transform_service.png alt="Diagram of the Transform service architecture"></p><p>To use the Transform service, Docker and Docker Compose must be available on the machine that starts the service.</p><p>The Transform service has the following primary use cases:</p><ul><li><p>Perform expansion of cross-language transforms without install [...] +in other SDKs without requiring you to install runtimes for the implementation languages of those SDKs. +For example, with the Transform service, a Beam Python pipeline can use the Google Cloud Java I/O transforms and Java Kafka I/O transforms +without a local Java runtime installation.</p></li><li><p>Upgrade transforms without upgrading the Apache Beam SDK version.</p><p>Use the Transform service to upgrade the Beam SDK versions of individual transforms used by Beam pipelines without upgrading the Beam version of the pipeline. +This feature is currently in development. For more details, see +<a href=https://github.com/apache/beam/issues/27943>GitHub issue #27943: Upgrade transforms without upgrading the pipeline using the Transform Service</a>.</p></li></ul><h3 id=transform-service-usage>15.1 Use the Transform service</h3><p>In some cases, Apache Beam SDKs automatically start the Transform service, such as in the following scenarios:</p><ul><li><p>The Java <a href=https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensi [...] +starts the Transform service when a Python runtime isn’t available locally, but Docker is.</p></li><li><p>The Apache Beam Python multi-language wrappers might automatically start the Transform service when you’re using Java transforms, a Java language runtime isn’t available locally, and Docker is available locally.</p></li></ul><p>To manually start a Transform service instance by using utilities provided with the Apache Beam SDKs, use the following commands.</p><div cl [...] +<a href=/documentation/transforms/python/elementwise/runinference/>RunInference</a> and +<a href=/documentation/dsls/dataframes/overview/>DataFrame</a> transforms.</p></li></ul><p>For a more comprehensive list of available transforms, see the +<a href=https://cwiki.apache.org/confluence/display/BEAM/Transform+Service>Transform service</a> developer guide.</p><div class=feedback><p class=update>Last updated on 2023/08/17</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div>< [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index b71fc49787c..2f55f8385cf 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/case-studies/octo/</loc><lastmod>2023-08-17T12:04:30-04:00</lastmod></url><url><loc>/blog/beam-2.49.0/</loc><lastmod>2023-08-17T12:04:30-04:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2023-08-17T12:04:30-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2023-08-17T12:04:30-04:00</lastmod></url><url><loc> [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/case-studies/octo/</loc><lastmod>2023-08-17T14:24:27-07:00</lastmod></url><url><loc>/blog/beam-2.49.0/</loc><lastmod>2023-08-17T14:24:27-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2023-08-17T14:24:27-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2023-08-17T14:24:27-07:00</lastmod></url><url><loc> [...] \ No newline at end of file