This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 9e044312424 Reorganizing the ML content (#25700)
9e044312424 is described below
commit 9e0443124243ab8c7458a9474f438e0f95e68636
Author: Rebecca Szper <[email protected]>
AuthorDate: Wed Mar 15 10:50:54 2023 -0700
Reorganizing the ML content (#25700)
* Restructuring the Beam ML content
* Restructuring the Beam ML content
* link fix
* Adding RunInference info to Get Started
* Removing duplicate content from About ML
* Adding a link to the RunInference table
* Adding a prediction and inference overview page
* Adding a prediction and inference overview page
---
.../site/content/en/documentation/ml/about-ml.md | 154 +++++++++++++++++++++
.../content/en/documentation/ml/data-processing.md | 2 +-
.../en/documentation/ml/inference-overview.md | 64 +++++++++
.../en/documentation/ml/large-language-modeling.md | 2 +-
.../site/content/en/documentation/ml/overview.md | 147 +++++++++++---------
.../documentation/sdks/python-machine-learning.md | 61 +++++---
.../python/elementwise/runinference-pytorch.md | 63 +++++++++
.../python/elementwise/runinference-sklearn.md | 64 +++++++++
.../transforms/python/elementwise/runinference.md | 86 +++---------
.../partials/section-menu/en/documentation.html | 64 +++++++--
10 files changed, 546 insertions(+), 161 deletions(-)
diff --git a/website/www/site/content/en/documentation/ml/about-ml.md
b/website/www/site/content/en/documentation/ml/about-ml.md
new file mode 100644
index 00000000000..766002ec227
--- /dev/null
+++ b/website/www/site/content/en/documentation/ml/about-ml.md
@@ -0,0 +1,154 @@
+---
+title: "About Beam ML"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# About Beam ML
+
+<table>
+ <tr>
+ <td>
+ <a>
+ {{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+ </a>
+ </td>
+ <td>
+ <a target="_blank" class="button"
+
href="https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/extensions/python/transforms/RunInference.html";>
+ <img src="https://beam.apache.org/images/logos/sdks/java.png";
width="20px" height="30px"
+ alt="Javadoc" />
+ Javadoc
+ </a>
+ </td>
+ </tr>
+</table>
+
+You can use Apache Beam to:
+
+* Process large volumes of data, both for preprocessing and for inference.
+* Experiment with your data during the exploration phase of your project and
provides a seamless transition when
+ upscaling your data pipelines as part of your MLOps ecosystem in a
production environment.
+* Run your model in production on a varying data load, both in batch and
streaming.
+
+## AI/ML workloads
+
+You can use Apache Beam for data validation, data preprocessing, model
validation, and model deployment/inference.
+
+
+
+1. Data ingestion: Incoming new data is stored in your file system or
database, or it's published to a messaging queue.
+2. **Data validation**: After you receieve your data, check the quality of
your data. For example, you might want to detect outliers and calculate
standard deviations and class distributions.
+3. **Data preprocessing**: After you validate your data, transform the data so
that it is ready to use to train your model.
+4. Model training: When your data is ready, you can start training your AI/ML
model. This step is typically repeated multiple times, depending on the quality
of your trained model.
+5. Model validation: Before you deploy your new model, validate its
performance and accuracy.
+6. **Model deployment**: Deploy your model, using it to run inference on new
or existing data.
+
+To keep your model up to date and performing well as your data grows and
evolves, run these steps multiple times. In addition, you can apply MLOps to
your project to automate the AI/ML workflows throughout the model and data
lifecycle. Use orchestrators to automate this flow and to handle the transition
between the different building blocks in your project.
+
+## Use RunInference
+
+The recommended way to implement inference in Apache Beam is by using the
[RunInference API](/documentation/sdks/python-machine-learning/). RunInference
takes advantage of existing Apache Beam concepts, such as the `BatchElements`
transform and the `Shared` class, to enable you to use models in your pipelines
to create transforms optimized for machine learning inferences. The ability to
create arbitrarily complex workflow graphs also allows you to build multi-model
pipelines.
+
+You can integrate your model in your pipeline by using the corresponding model
handlers. A `ModelHandler` is an object that wraps the underlying model and
allows you to configure its parameters. Model handlers are available for
PyTorch, scikit-learn, and TensorFlow. Examples of how to use RunInference for
PyTorch, scikit-learn, and TensorFlow are shown in this
[notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb).
+
+Because GPUs can process multiple computations simultaneously, they are
optimized for training artificial intelligence and deep learning models.
RunInference also allows you to use GPUs for significant inference speedup. An
example of how to use RunInference with GPUs is demonstrated on the
[RunInference metrics](/documentation/ml/runinference-metrics) page.
+
+RunInference takes advantage of existing Apache Beam concepts, such as the
`BatchElements` transform and the `Shared` class, to enable you to use models
in your pipelines to create transforms optimized for machine learning
inferences. The ability to create arbitrarily complex workflow graphs also
allows you to build multi-model pipelines.
+
+### BatchElements PTransform
+
+To take advantage of the optimizations of vectorized inference that many
models implement, we added the `BatchElements` transform as an intermediate
step before making the prediction for the model. This transform batches
elements together. The batched elements are then applied with a transformation
for the particular framework of RunInference. For example, for numpy
`ndarrays`, we call `numpy.stack()`, and for torch `Tensor` elements, we call
`torch.stack()`.
+
+To customize the settings for `beam.BatchElements`, in `ModelHandler`,
override the `batch_elements_kwargs` function. For example, use
`min_batch_size` to set the lowest number of elements per batch or
`max_batch_size` to set the highest number of elements per batch.
+
+For more information, see the [`BatchElements` transform
documentation](https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.util.html#apache_beam.transforms.util.BatchElements).
+
+### Shared helper class
+
+Using the `Shared` class within the RunInference implementation makes it
possible to load the model only once per process and share it with all DoFn
instances created in that process. This feature reduces memory consumption and
model loading time. For more information, see the
+[`Shared` class
documentation](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/utils/shared.py#L20).
+
+### Multi-model pipelines
+
+The RunInference API can be composed into multi-model pipelines. Multi-model
pipelines can be useful for A/B testing or for building out cascade models made
up of models that perform tokenization, sentence segmentation, part-of-speech
tagging, named entity extraction, language detection, coreference resolution,
and more. For more information, see [Multi-model
pipelines](https://beam.apache.org/documentation/ml/multi-model-pipelines/).
+
+### Use pre-trained models
+
+The section provides requirements for using pre-trained models with PyTorch,
Scikit-learn, and Tensorflow.
+
+#### PyTorch
+
+You need to provide a path to a file that contains the model's saved weights.
This path must be accessible by the pipeline. To use pre-trained models with
the RunInference API and the PyTorch framework, complete the following steps:
+
+1. Download the pre-trained weights and host them in a location that the
pipeline can access.
+2. Pass the path of the model weights to the PyTorch `ModelHandler` by using
the following code: `state_dict_path=<path_to_weights>`.
+
+See [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch.ipynb)
+that illustrates running PyTorch models with Apache Beam.
+
+#### Scikit-learn
+
+You need to provide a path to a file that contains the pickled Scikit-learn
model. This path must be accessible by the pipeline. To use pre-trained models
with the RunInference API and the Scikit-learn framework, complete the
following steps:
+
+1. Download the pickled model class and host it in a location that the
pipeline can access.
+2. Pass the path of the model to the Sklearn `ModelHandler` by using the
following code:
+ `model_uri=<path_to_pickled_file>` and `model_file_type: <ModelFileType>`,
where you can specify
+ `ModelFileType.PICKLE` or `ModelFileType.JOBLIB`, depending on how the
model was serialized.
+
+See [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_sklearn.ipynb)
+that illustrates running Scikit-learn models with Apache Beam.
+
+#### TensorFlow
+
+To use TensorFlow with the RunInference API, you need to do the following:
+
+* Use `tfx_bsl` version 1.10.0 or later.
+* Create a model handler using
`tfx_bsl.public.beam.run_inference.CreateModelHandler()`.
+* Use the model handler with the
[`apache_beam.ml.inference.base.RunInference`](/releases/pydoc/current/apache_beam.ml.inference.base.html)
transform.
+
+See [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow.ipynb)
+that illustrates running TensorFlow models with Apache Beam and tfx-bsl.
+
+## Custom Inference
+
+The RunInference API doesn't currently support making remote inference calls
using, for example, the Natural Language API or the Cloud Vision API.
Therefore, in order to use these remote APIs with Apache Beam, you need to
write custom inference calls. The [Remote inference in Apache Beam
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/custom_remote_inference.ipynb)
shows how to implement a custom remote inference call using `beam.DoFn`. When
you implement [...]
+
+* API quotas and the heavy load you might incur on your external API. To
optimize the calls to an external API, you can confgure `PipelineOptions` to
limit the parallel calls to the external remote API.
+
+* Be prepared to encounter, identify, and handle failure as gracefully as
possible. Use techniques like exponential backoff and dead-letter queues
(unprocessed messages queues).
+
+* When running inference with an external API, batch your input together to
allow for more efficient execution.
+
+* Consider monitoring and measuring the performance of a pipeline when
deploying, because monitoring can provide insight into the status and health of
the application.
+
+### Use custom models
+
+If you would like to use a model that isn't specified by one of the supported
frameworks, the RunInference API is designed flexibly to allow you to use any
custom machine learning models.
+You only need to create your own `ModelHandler` or `KeyedModelHandler` with
logic to load your model and use it to run the inference.
+
+A simple example can be found in [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_custom_inference.ipynb).
+The `load_model` method shows how to load the model using a popular `spaCy`
package while `run_inference` shows how to run the inference on a batch of
examples.
+
+## Model validation
+
+Model validation allows you to benchmark your model’s performance against a
previously unseen dataset. You can extract chosen metrics, create
visualizations, log metadata, and compare the performance of different models
with the end goal of validating whether your model is ready to deploy. Beam
provides support for running model evaluation on a TensorFlow model directly
inside your pipeline.
+
+The [ML model evaluation](/documentation/ml/model-evaluation) page shows how
to integrate model evaluation as part of your pipeline by using [TensorFlow
Model Analysis (TFMA)](https://www.tensorflow.org/tfx/guide/tfma).
+
+## Related links
+
+* [RunInference public
codelab](https://colab.sandbox.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_basic.ipynb)
+* [RunInference
notebooks](https://github.com/apache/beam/tree/master/examples/notebooks/beam-ml)
+* [RunInference
benchmarks](http://s.apache.org/beam-community-metrics/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1)
\ No newline at end of file
diff --git a/website/www/site/content/en/documentation/ml/data-processing.md
b/website/www/site/content/en/documentation/ml/data-processing.md
index 19163f27134..9b6ec5117a5 100755
--- a/website/www/site/content/en/documentation/ml/data-processing.md
+++ b/website/www/site/content/en/documentation/ml/data-processing.md
@@ -1,5 +1,5 @@
---
-title: "Overview"
+title: "Data processing"
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
diff --git a/website/www/site/content/en/documentation/ml/inference-overview.md
b/website/www/site/content/en/documentation/ml/inference-overview.md
new file mode 100644
index 00000000000..46aaa139e55
--- /dev/null
+++ b/website/www/site/content/en/documentation/ml/inference-overview.md
@@ -0,0 +1,64 @@
+---
+title: "Prediction and inference overview"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Prediction and inference
+
+
+Beam provides different ways to implement inference as part of your pipeline.
You can run your ML model directly in your pipeline and apply it on big scale
datasets, both in batch and streaming pipelines.
+
+## Use the RunInference API
+
+<table>
+ <tr>
+ <td>
+ <a>
+ {{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+ </a>
+ </td>
+ <td>
+ <a target="_blank" class="button"
+
href="https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/extensions/python/transforms/RunInference.html";>
+ <img src="https://beam.apache.org/images/logos/sdks/java.png";
width="20px" height="30px"
+ alt="Javadoc" />
+ Javadoc
+ </a>
+ </td>
+ </tr>
+</table>
+
+The RunInfernce API is available with the Beam Python SDK versions 2.40.0 and
later. You can use Apache Beam with the RunInference API to use machine
learning (ML) models to do local and remote inference with batch and streaming
pipelines. Starting with Apache Beam 2.40.0, PyTorch and Scikit-learn
frameworks are supported. Tensorflow models are supported through `tfx-bsl`.
For more deatils about using RunInference with Python, see [Machine Learning
with Python](/documentation/sdks/python [...]
+
+The RunInference API is available with the Beam Java SDK versions 2.41.0 and
later through Apache Beam's [Multi-language Pipelines
framework](/documentation/programming-guide/#multi-language-pipelines). For
information about the Java wrapper transform, see
[RunInference.java](https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/transforms/RunInference.java).
To try it out, see the [Java Sklearn Mnist Classification exa [...]
+
+You can create multiple types of transforms using the RunInference API: the
API takes multiple types of setup parameters from model handlers, and the
parameter type determines the model implementation.
+
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to use the RunInference transform | [Modify a Python pipeline to use
an ML
model](/documentation/sdks/python-machine-learning/#modify-a-python-pipeline-to-use-an-ml-model)
|
+| I want to use RunInference with PyTorch | [Use RunInference with
PyTorch](/documentation/transforms/python/elementwise/runinference-pytorch/) |
+| I want to use RunInference with Sklearn | [Use RunInference with
Sklearn](/documentation/transforms/python/elementwise/runinference-sklearn/) |
+| I want to use pre-trained models (PyTorch, Scikit-learn, or TensorFlow) |
[Use pre-trained models](/documentation/ml/about-ml/#use-pre-trained-models) |:
+{{< /table >}}
+
+## Use cases
+
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to build a pipeline with multiple models | [Multi-Model
Pipelines](/documentation/ml/multi-model-pipelines) |
+| I want to build a custom model handler with TensorRT | [Use TensorRT with
RunInference](/documentation/ml/tensorrt-runinference) |
+| I want to use LLM inference | [Large Language Model
Inference](/documentation/ml/large-language-modeling/) |:
+{{< /table >}}
\ No newline at end of file
diff --git
a/website/www/site/content/en/documentation/ml/large-language-modeling.md
b/website/www/site/content/en/documentation/ml/large-language-modeling.md
index 84e9edaea3d..9148f5c28b9 100644
--- a/website/www/site/content/en/documentation/ml/large-language-modeling.md
+++ b/website/www/site/content/en/documentation/ml/large-language-modeling.md
@@ -15,7 +15,7 @@ See the License for the specific language governing
permissions and
limitations under the License.
-->
-# RunInference
+# Large Language Model Inference in Beam
In Apache Beam 2.40.0, Beam introduced the RunInference API, which lets you
deploy a machine learning model in a Beam pipeline. A `RunInference` transform
performs inference on a `PCollection` of examples using a machine learning (ML)
model. The transform outputs a PCollection that contains the input examples and
output predictions. For more information, see RunInference
[here](/documentation/transforms/python/elementwise/runinference/). You can
also find [inference examples on GitHub](h [...]
diff --git a/website/www/site/content/en/documentation/ml/overview.md
b/website/www/site/content/en/documentation/ml/overview.md
index feec2c4e807..8e5641ccbff 100644
--- a/website/www/site/content/en/documentation/ml/overview.md
+++ b/website/www/site/content/en/documentation/ml/overview.md
@@ -1,13 +1,11 @@
---
-title: "Overview"
+title: "Get Started with ML"
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
-
http://www.apache.org/licenses/LICENSE-2.0
-
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -15,93 +13,118 @@ See the License for the specific language governing
permissions and
limitations under the License.
-->
-# AI/ML pipelines
-
-Beam <3 machine learning. Being productive and successful as a machine
learning practitioner is often dependent on your ability to efficiently
leverage large volumes of data in a way that is uniquely tailored to your
resources, requirements, and budget. Whether starting your next AI/ML project
or upscaling an existing project, consider adding Apache Beam to your project.
-
-* Apache Beam enables you to process large volumes of data, both for
preprocessing and for inference.
-* It allows you to experiment with your data during the exploration phase of
your project and provides a seamless transition when
- upscaling your data pipelines as part of your MLOps ecosystem in a
production environment.
-* It enables you to run your model in production on a varying data load, both
in batch and streaming.
+# Get started with AI/ML pipelines
+
+<table>
+ <tr>
+ <td>
+ <a>
+ {{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+ </a>
+ </td>
+ <td>
+ <a target="_blank" class="button"
+
href="https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/extensions/python/transforms/RunInference.html";>
+ <img src="https://beam.apache.org/images/logos/sdks/java.png";
width="20px" height="30px"
+ alt="Javadoc" />
+ Javadoc
+ </a>
+ </td>
+ </tr>
+</table>
+
+Being productive and successful as a machine learning practitioner is often
dependent on your ability to efficiently leverage large volumes of data in a
way that is uniquely tailored to your resources, requirements, and budget.
Whether starting your next AI/ML project or upscaling an existing project,
consider adding Apache Beam to your project.
<iframe class="video video--medium-size" width="560" height="315"
src="https://www.youtube.com/embed/ga2TNdrFRoU"; frameborder="0"
allowfullscreen></iframe>
-## AI/ML workloads
-
-Let’s take a look at the different building blocks that we need to create an
end-to-end AI/ML use case and where Apache Beam can help.
+## Use Beam ML
-
+I want use Beam ML to do:
-1. Data ingestion: Incoming new data is stored in your file system or
database, or it's published to a messaging queue.
-2. **Data validation**: After you receieve your data, check the quality of
your data. For example, you might want to detect outliers and calculate
standard deviations and class distributions.
-3. **Data preprocessing**: After you validate your data, transform the data so
that it is ready to use to train your model.
-4. Model training: When your data is ready, you can start training your AI/ML
model. This step is typically repeated multiple times, depending on the quality
of your trained model.
-5. **Model validation**: Before you deploy your new model, validate its
performance and accuracy.
-6. **Model deployment**: Deploy your model, using it to run inference on new
or existing data.
+* [Prediction and inference](#prediction-and-inference)
+* [Data processing](#data-processing)
+* [Workflow orchestration](#workflow-orchestration)
+* [Model training](#model-training)
+* [Anomaly detection](#use-cases)
-To keep your model up to date and performing well as your data grows and
evolves, run these steps multiple times. In addition, you can apply MLOps to
your project to automate the AI/ML workflows throughout the model and data
lifecycle. Use orchestrators to automate this flow and to handle the transition
between the different building blocks in your project.
-
-You can use Apache Beam for data validation, data preprocessing, model
validation, and model deployment/inference. The next section examines these
building blocks in more detail and explores how they can be orchestrated.
-
-## Data processing
-
-You can use Apache Beam for data validation and preprocessing by setting up
data pipelines that transform your data and output metrics computed from your
data. Beam has a rich set of [I/O connectors](/documentation/io/built-in/) for
ingesting and writing data, which allows you to integrate it with your existing
file system, database, or messaging queue.
-When developing your ML model, you can also first explore your data with the
[Beam DataFrame API](/documentation/dsls/dataframes/overview/). The DataFrom
API lets you identify and implement the required preprocessing steps, making it
easier for you to move your pipeline to production.
-
-Steps executed during preprocessing often also need to be applied before
running inference, in which case you can use the same Beam implementation
twice. Lastly, when you need to do postprocessing after running inference,
Apache Beam allows you to incoporate the postprocessing into your model
inference pipeline.
-
-Further reading:
-* [AI/ML pipelines in Beam: data processing](/documentation/ml/data-processing)
-
-## Inference
+## Prediction and inference
Beam provides different ways to implement inference as part of your pipeline.
You can run your ML model directly in your pipeline and apply it on big scale
datasets, both in batch and streaming pipelines.
### RunInference
-The recommended way to implement inference is by using the [RunInference
API](/documentation/sdks/python-machine-learning/). RunInference takes
advantage of existing Apache Beam concepts, such as the `BatchElements`
transform and the `Shared` class, to enable you to use models in your pipelines
to create transforms optimized for machine learning inferences. The ability to
create arbitrarily complex workflow graphs also allows you to build multi-model
pipelines.
+The RunInfernce API is available with the Beam Python SDK versions 2.40.0 and
later. You can use Apache Beam with the RunInference API to use machine
learning (ML) models to do local and remote inference with batch and streaming
pipelines. Starting with Apache Beam 2.40.0, PyTorch and Scikit-learn
frameworks are supported. Tensorflow models are supported through `tfx-bsl`.
For more deatils about using RunInference with Python, see [Machine Learning
with Python](/documentation/sdks/python [...]
-You can integrate your model in your pipeline by using the corresponding model
handlers. A `ModelHandler` is an object that wraps the underlying model and
allows you to configure its parameters. Model handlers are available for
PyTorch, scikit-learn, and TensorFlow. Examples of how to use RunInference for
PyTorch, scikit-learn, and TensorFlow are shown in the [RunInference
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_skl
[...]
+The RunInference API is available with the Beam Java SDK versions 2.41.0 and
later through Apache Beam's [Multi-language Pipelines
framework](/documentation/programming-guide/#multi-language-pipelines). For
information about the Java wrapper transform, see
[RunInference.java](https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/transforms/RunInference.java).
To try it out, see the [Java Sklearn Mnist Classification exa [...]
-Because they can process multiple computations simultaneously, GPUs are
optimized for training artificial intelligence and deep learning models.
RunInference also allows you to use GPUs for significant inference speedup. An
example of how to use RunInference with GPUs is demonstrated on the
[RunInference metrics](/documentation/ml/runinference-metrics) page.
+You can create multiple types of transforms using the RunInference API: the
API takes multiple types of setup parameters from model handlers, and the
parameter type determines the model implementation.
-Another usecase of running machine learning models is to run them on hardware
devices. [Nvidia TensorRT](https://developer.nvidia.com/tensorrt) is a machine
learning framework used to run inference on Nvidia hardware. See [TensorRT
Inference](/documentation/ml/tensorrt-runinference) for an example of a
pipeline that uses TensorRT and Beam with the RunInference transform and a
BERT-based text classification model.
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to use the RunInference transform | [Modify a Python pipeline to use
an ML
model](/documentation/sdks/python-machine-learning/#modify-a-python-pipeline-to-use-an-ml-model)
|
+| I want to use RunInference with PyTorch | [Use RunInference with
PyTorch](/documentation/transforms/python/elementwise/runinference-pytorch/) |
+| I want to use RunInference with Sklearn | [Use RunInference with
Sklearn](/documentation/transforms/python/elementwise/runinference-sklearn/) |
+| I want to use pre-trained models (PyTorch, Scikit-learn, or TensorFlow) |
[Use pre-trained models](/documentation/ml/about-ml/#use-pre-trained-models) |:
+{{< /table >}}
-### Custom Inference
+### Prediction and inference examples
-The RunInference API doesn't currently support making remote inference calls
using, for example, the Natural Language API or the Cloud Vision API.
Therefore, in order to use these remote APIs with Apache Beam, you need to
write custom inference calls. The [Remote inference in Apache Beam
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/custom_remote_inference.ipynb)
shows how to implement a custom remote inference call using `beam.DoFn`. When
you implement [...]
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to build a pipeline with multiple models | [Multi-Model
Pipelines](/documentation/ml/multi-model-pipelines) |
+| I want to build a custom model handler with TensorRT | [Use TensorRT with
RunInference](/documentation/ml/tensorrt-runinference) |
+| I want to use LLM inference | [Large Language Model
Inference](/documentation/ml/large-language-modeling/) |:
+{{< /table >}}
-* API quotas and the heavy load you might incur on your external API. To
optimize the calls to an external API, you can confgure `PipelineOptions` to
limit the parallel calls to the external remote API.
+## Data processing
-* Be prepared to encounter, identify, and handle failure as gracefully as
possible. Use techniques like exponential backoff and dead-letter queues
(unprocessed messages queues).
+You can use Apache Beam for data validation and preprocessing by setting up
data pipelines that transform your data and output metrics computed from your
data. Beam has a rich set of [I/O connectors](/documentation/io/built-in/) for
ingesting and writing data, which allows you to integrate it with your existing
file system, database, or messaging queue.
-* When running inference with an external API, batch your input together to
allow for more efficient execution.
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to explore my data | [Data Processing
Example](/documentation/ml/data-processing) |:
+{{< /table >}}
-* Consider monitoring and measuring the performance of a pipeline when
deploying, because monitoring can provide insight into the status and health of
the application.
-## Model validation
+## Workflow orchestration
-Model validation allows you to benchmark your model’s performance against a
previously unseen dataset. You can extract chosen metrics, create
visualizations, log metadata, and compare the performance of different models
with the end goal of validating whether your model is ready to deploy. Beam
provides support for running model evaluation on a TensorFlow model directly
inside your pipeline.
+In order to automate and track the AI/ML workflows throughout your project,
you can use orchestrators such as [Kubeflow
pipelines](https://www.kubeflow.org/docs/components/pipelines/introduction/)
(KFP) or [TensorFlow Extended](https://www.tensorflow.org/tfx) (TFX). These
orchestrators automate your different building blocks and handle the
transitions between them.
-Further reading:
-* [ML model evaluation](/documentation/ml/model-evaluation): Illustrates how
to integrate model evaluation as part of your pipeline by using [TensorFlow
Model Analysis (TFMA)](https://www.tensorflow.org/tfx/guide/tfma).
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to use ML-OPS workflow orchestrators | [Workflow
Orchestration](/documentation/ml/orchestration/) |
+| I want to use Beam with TensorFlow Extended (TFX) | [Tensorflow Extended
(TFX)](/documentation/ml/orchestration/#tensorflow-extended-tfx) |
+| I want to use Beam with Kubeflow | [Kubeflow pipelines
(KFP)](/documentation/ml/orchestration/#kubeflow-pipelines-kfp) |:
+{{< /table >}}
-## Orchestrators
+When you use Apache Beam as one of the building blocks in your project, these
orchestrators are able to launch your Apache Beam job and to keep track of the
input and output of your pipeline. These tasks are essential when moving your
AI/ML solution into production, because they allow you to handle your model and
data over time and improve the quality and reproducibility of results.
-In order to automate and track the AI/ML workflows throughout your project,
you can use orchestrators such as [Kubeflow
pipelines](https://www.kubeflow.org/docs/components/pipelines/introduction/)
(KFP) or [TensorFlow Extended](https://www.tensorflow.org/tfx) (TFX). These
orchestrators automate your different building blocks and handle the
transitions between them.
+## Model training
-When you use Apache Beam as one of the building blocks in your project, these
orchestrators are able to launch your Apache Beam job and to keep track of the
input and output of your pipeline. These tasks are essential when moving your
AI/ML solution into production, because they allow you to handle your model and
data over time and improve the quality and reproducibility of results.
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to use per-entity training | [Per Entity
Training](/documentation/ml/per-entity-training) |
+| I want to cluster text | [Online Clustering
Example](/documentation/ml/online-clustering) |
+| I want to benchmark model performance | [ML Model
Evaluation](/documentation/ml/model-evaluation/) |:
+{{< /table >}}
-Further reading:
-* [ML Workflow Orchestration](/documentation/ml/orchestration): Illustrates
how to orchestrate ML workflows consisting of multiple steps by using Kubeflow
Pipelines and Tensorflow Extended.
+## Use cases
-## Examples
+{{< table >}}
+| Task | Example |
+| ------- | ---------------|
+| I want to build an anomaly detection pipeline | [Anomaly Detection
Example](/documentation/ml/anomaly-detection/) |:
+{{< /table >}}
-You can find examples of end-to-end AI/ML pipelines for several use cases:
+## Reference
-* [Multi model pipelines in Beam](/documentation/ml/multi-model-pipelines):
Explains how multi-model pipelines work and gives an overview of what you need
to know to build one using the RunInference API.
-* [Online Clustering in Beam](/documentation/ml/online-clustering):
Demonstrates how to set up a real-time clustering pipeline that can read text
from Pub/Sub, convert the text into an embedding using a transformer-based
language model with the RunInference API, and cluster the text using BIRCH with
stateful processing.
-* [Anomaly Detection in Beam](/documentation/ml/anomaly-detection):
Demonstrates how to set up an anomaly detection pipeline that reads text from
Pub/Sub in real time and then detects anomalies using a trained HDBSCAN
clustering model with the RunInference API.
-* [Large Language Model Inference in
Beam](/documentation/ml/large-language-modeling): Demonstrates a pipeline that
uses RunInference to perform translation with the T5 language model which
contains 11 billion parameters.
-* [Per Entity Training in Beam](/documentation/ml/per-entity-training):
Demonstrates a pipeline that trains a Decision Tree Classifier per education
level for predicting if the salary of a person is >= 50k.
+* [RunInference metrics](/documentation/ml/runinference-metrics/)
+* [ML model evaluation](/documentation/ml/model-evaluation/)
+* [RunInference public
codelab](https://colab.sandbox.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_basic.ipynb)
+* [RunInference
notebooks](https://github.com/apache/beam/tree/master/examples/notebooks/beam-ml)
\ No newline at end of file
diff --git
a/website/www/site/content/en/documentation/sdks/python-machine-learning.md
b/website/www/site/content/en/documentation/sdks/python-machine-learning.md
index 8c07e4b7418..7e6474f451b 100644
--- a/website/www/site/content/en/documentation/sdks/python-machine-learning.md
+++ b/website/www/site/content/en/documentation/sdks/python-machine-learning.md
@@ -16,7 +16,7 @@ See the License for the specific language governing
permissions and
limitations under the License.
-->
-# Machine Learning
+# Machine Learning with Python
<table>
<tr>
@@ -36,10 +36,17 @@ limitations under the License.
</tr>
</table>
+
You can use Apache Beam with the RunInference API to use machine learning (ML)
models to do local and remote inference with batch and streaming pipelines.
Starting with Apache Beam 2.40.0, PyTorch and Scikit-learn frameworks are
supported. Tensorflow models are supported through tfx-bsl.
You can create multiple types of transforms using the RunInference API: the
API takes multiple types of setup parameters from model handlers, and the
parameter type determines the model implementation.
+For more infomation about machine learning with Apache Beam, see:
+
+* [Get started with AI/ML](/documentation/ml/overview)
+* [About Beam ML](/documentation/ml/about-ml)
+* [RunInference
notebooks](https://github.com/apache/beam/tree/master/examples/notebooks/beam-ml)
+
## Why use the RunInference API?
RunInference takes advantage of existing Apache Beam concepts, such as the
`BatchElements` transform and the `Shared` class, to enable you to use models
in your pipelines to create transforms optimized for machine learning
inferences. The ability to create arbitrarily complex workflow graphs also
allows you to build multi-model pipelines.
@@ -60,8 +67,10 @@ Using the `Shared` class within the RunInference
implementation makes it possibl
### Multi-model pipelines
The RunInference API can be composed into multi-model pipelines. Multi-model
pipelines can be useful for A/B testing or for building out cascade models made
up of models that perform tokenization, sentence segmentation, part-of-speech
tagging, named entity extraction, language detection, coreference resolution,
and more.
+For more information about multi-model pipelines, see
+[Multi-model pipelines](/documentation/ml/multi-model-pipelines/).
-## Modify a pipeline to use an ML model
+## Modify a Python pipeline to use an ML model
To use the RunInference transform, add the following code to your pipeline:
@@ -82,11 +91,12 @@ from apache_beam.ml.inference.pytorch_inference import
PytorchModelHandlerTensor
from apache_beam.ml.inference.pytorch_inference import
PytorchModelHandlerKeyedTensor
from tfx_bsl.public.beam.run_inference import CreateModelHandler
```
-### Use pre-trained models
+
+## Use pre-trained models
The section provides requirements for using pre-trained models with PyTorch
and Scikit-learn
-#### PyTorch
+### PyTorch
You need to provide a path to a file that contains the model's saved weights.
This path must be accessible by the pipeline. To use pre-trained models with
the RunInference API and the PyTorch framework, complete the following steps:
@@ -96,7 +106,7 @@ You need to provide a path to a file that contains the
model's saved weights. Th
See [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch.ipynb)
that illustrates running PyTorch models with Apache Beam.
-#### Scikit-learn
+### Scikit-learn
You need to provide a path to a file that contains the pickled Scikit-learn
model. This path must be accessible by the pipeline. To use pre-trained models
with the RunInference API and the Scikit-learn framework, complete the
following steps:
@@ -108,7 +118,7 @@ You need to provide a path to a file that contains the
pickled Scikit-learn mode
See [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_sklearn.ipynb)
that illustrates running Scikit-learn models with Apache Beam.
-#### TensorFlow
+### TensorFlow
To use TensorFlow with the RunInference API, you need to do the following:
@@ -119,7 +129,7 @@ To use TensorFlow with the RunInference API, you need to do
the following:
See [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow.ipynb)
that illustrates running TensorFlow models with Apache Beam and tfx-bsl.
-### Use custom models
+## Use custom models
If you would like to use a model that isn't specified by one of the supported
frameworks, the RunInference API is designed flexibly to allow you to use any
custom machine learning models.
You only need to create your own `ModelHandler` or `KeyedModelHandler` with
logic to load your model and use it to run the inference.
@@ -127,11 +137,11 @@ You only need to create your own `ModelHandler` or
`KeyedModelHandler` with logi
A simple example can be found in [this
notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_custom_inference.ipynb).
The `load_model` method shows how to load the model using a popular `spaCy`
package while `run_inference` shows how to run the inference on a batch of
examples.
-### Use multiple models
+## Use multiple models
-You can also use the RunInference transform to add multiple inference models
to your pipeline.
+You can also use the RunInference transform to add multiple inference models
to your Python pipeline.
-#### A/B Pattern
+### A/B Pattern
```
with pipeline as p:
@@ -142,7 +152,7 @@ with pipeline as p:
Where `model_handler_A` and `model_handler_B` are the model handler setup code.
-#### Cascade Pattern
+### Cascade Pattern
```
with pipeline as p:
@@ -153,7 +163,7 @@ with pipeline as p:
Where `model_handler_A` and `model_handler_B` are the model handler setup code.
-#### Use Resource Hints for Different Model Requirements
+### Use Resource Hints for Different Model Requirements
When using multiple models in a single pipeline, different models may have
different memory or worker SKU requirements.
Resource hints allow you to provide information to a runner about the compute
resource requirements for each step in your
@@ -175,7 +185,7 @@ with pipeline as p:
For more information on resource hints, see [Resource
hints](/documentation/runtime/resource-hints/).
-### Use a keyed ModelHandler
+## Use a keyed ModelHandler
If a key is attached to the examples, wrap the `KeyedModelHandler` around the
`ModelHandler` object:
@@ -195,7 +205,7 @@ If you are unsure if your data is keyed, you can also use
`MaybeKeyedModelHandle
For more information, see
[`KeyedModelHander`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.KeyedModelHandler).
-### Use the PredictionResults object
+## Use the PredictionResults object
When doing a prediction in Apache Beam, the output `PCollection` includes both
the keys of the input examples and the inferences. Including both these items
in the output allows you to find the input that determined the predictions.
@@ -230,13 +240,14 @@ For more information, see the [`PredictionResult`
documentation](https://github.
## Run a machine learning pipeline
-For detailed instructions explaining how to build and run a pipeline that uses
ML models, see the
+For detailed instructions explaining how to build and run a Python pipeline
that uses ML models, see the
[Example RunInference API
pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference)
on GitHub.
## Beam Java SDK support
The RunInference API is available with the Beam Java SDK versions 2.41.0 and
later through Apache Beam's [Multi-language Pipelines
framework](/documentation/programming-guide/#multi-language-pipelines). For
information about the Java wrapper transform, see
[RunInference.java](https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/transforms/RunInference.java).
To try it out, see the [Java Sklearn Mnist Classification exa [...]
+
## Troubleshooting
If you run into problems with your pipeline or job, this section lists issues
that you might encounter and provides suggestions for how to fix them.
@@ -260,7 +271,7 @@ Use elements of the same size or resize the inputs. For
computer vision applicat
**Option 2: Disable batching**
Disable batching by overriding the `batch_elements_kwargs` function in your
ModelHandler and setting the maximum batch size (`max_batch_size`) to one:
`max_batch_size=1`. For more information, see
-[BatchElements
PTransforms](/documentation/sdks/python-machine-learning/#batchelements-ptransform).
For an example, see our [language modeling
example](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_language_modeling.py).
+[BatchElements
PTransforms](/documentation/ml/about-ml/#batchelements-ptransform). For an
example, see our [language modeling
example](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_language_modeling.py).
## Related links
@@ -270,4 +281,20 @@ Disable batching by overriding the `batch_elements_kwargs`
function in your Mode
* [RunInference
notebooks](https://github.com/apache/beam/tree/master/examples/notebooks/beam-ml)
* [RunInference
benchmarks](http://s.apache.org/beam-community-metrics/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1)
-{{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+<table>
+ <tr>
+ <td>
+ <a>
+ {{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+ </a>
+ </td>
+ <td>
+ <a target="_blank" class="button"
+
href="https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/extensions/python/transforms/RunInference.html";>
+ <img src="https://beam.apache.org/images/logos/sdks/java.png";
width="20px" height="30px"
+ alt="Javadoc" />
+ Javadoc
+ </a>
+ </td>
+ </tr>
+</table>
diff --git
a/website/www/site/content/en/documentation/transforms/python/elementwise/runinference-pytorch.md
b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference-pytorch.md
new file mode 100644
index 00000000000..f0bfae7f4cc
--- /dev/null
+++
b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference-pytorch.md
@@ -0,0 +1,63 @@
+---
+title: "RunInference with PyTorch"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use RunInference with PyTorch
+
+<table>
+ <tr>
+ <td>
+ <a>
+ {{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+ </a>
+ </td>
+ </tr>
+</table>
+
+The following examples demonstrate how to to create pipelines that use the
Beam RunInference API and PyTorch.
+
+## Example 1: PyTorch unkeyed model
+
+In this example, we create a pipeline that uses a PyTorch RunInference
transform on unkeyed data.
+
+{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
+ class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
torch_unkeyed_model_handler >}}
+{{</ highlight >}}
+
+{{< paragraph class="notebook-skip" >}}
+Output:
+{{< /paragraph >}}
+{{< highlight class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
torch_unkeyed_model_handler >}}
+{{< /highlight >}}
+
+## Example 2: PyTorch keyed model
+
+In this example, we create a pipeline that uses a PyTorch RunInference
transform on keyed data.
+
+{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
+ class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
torch_keyed_model_handler >}}
+{{</ highlight >}}
+
+{{< paragraph class="notebook-skip" >}}
+Output:
+{{< /paragraph >}}
+
+{{< highlight class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
torch_keyed_model_handler >}}
+{{< /highlight >}}
diff --git
a/website/www/site/content/en/documentation/transforms/python/elementwise/runinference-sklearn.md
b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference-sklearn.md
new file mode 100644
index 00000000000..5e3fad1cf41
--- /dev/null
+++
b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference-sklearn.md
@@ -0,0 +1,64 @@
+---
+title: "RunInference with Sklearn"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use RunInference with Sklearn
+
+<table>
+ <tr>
+ <td>
+ <a>
+ {{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+ </a>
+ </td>
+ </tr>
+</table>
+
+The following examples demonstrate how to to create pipelines that use the
Beam RunInference API and Sklearn.
+
+## Example 1: Sklearn unkeyed model
+
+In this example, we create a pipeline that uses an SKlearn RunInference
transform on unkeyed data.
+
+{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
+ class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
sklearn_unkeyed_model_handler >}}
+{{</ highlight >}}
+
+{{< paragraph class="notebook-skip" >}}
+Output:
+{{< /paragraph >}}
+
+{{< highlight class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
sklearn_unkeyed_model_handler >}}
+{{< /highlight >}}
+
+## Example 2: Sklearn keyed model
+
+In this example, we create a pipeline that uses an SKlearn RunInference
transform on keyed data.
+
+{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
+ class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
sklearn_keyed_model_handler >}}
+{{</ highlight >}}
+
+{{< paragraph class="notebook-skip" >}}
+Output:
+{{< /paragraph >}}
+
+{{< highlight class="notebook-skip" >}}
+{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
sklearn_keyed_model_handler >}}
+{{< /highlight >}}
\ No newline at end of file
diff --git
a/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md
b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md
index 42455aa073a..39db1aa68d3 100644
---
a/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md
+++
b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md
@@ -19,7 +19,15 @@ limitations under the License.
{{< localstorage language language-py >}}
-{{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+<table>
+ <tr>
+ <td>
+ <a>
+ {{< button-pydoc path="apache_beam.ml.inference" class="RunInference" >}}
+ </a>
+ </td>
+ </tr>
+</table>
Uses models to do local and remote inference. A `RunInference` transform
performs inference on a `PCollection` of examples using a machine learning (ML)
model. The transform outputs a `PCollection` that contains the input examples
and output predictions.
@@ -29,74 +37,16 @@ See more [RunInference API pipeline
examples](https://github.com/apache/beam/tre
## Examples
-In the following examples, we explore how to create pipelines that use the
Beam RunInference API to make predictions based on models.
-
-### Example 1: PyTorch unkeyed model
-
-In this example, we create a pipeline that uses a PyTorch RunInference
transform on unkeyed data.
-
-{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
- class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
torch_unkeyed_model_handler >}}
-{{</ highlight >}}
-
-{{< paragraph class="notebook-skip" >}}
-Output:
-{{< /paragraph >}}
-{{< highlight class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
torch_unkeyed_model_handler >}}
-{{< /highlight >}}
-
-### Example 2: PyTorch keyed model
-
-In this example, we create a pipeline that uses a PyTorch RunInference
transform on keyed data.
-
-{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
- class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
torch_keyed_model_handler >}}
-{{</ highlight >}}
-
-{{< paragraph class="notebook-skip" >}}
-Output:
-{{< /paragraph >}}
-
-{{< highlight class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
torch_keyed_model_handler >}}
-{{< /highlight >}}
-
-### Example 3: Sklearn unkeyed model
-
-In this example, we create a pipeline that uses an SKlearn RunInference
transform on unkeyed data.
-
-{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
- class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
sklearn_unkeyed_model_handler >}}
-{{</ highlight >}}
-
-{{< paragraph class="notebook-skip" >}}
-Output:
-{{< /paragraph >}}
-
-{{< highlight class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
sklearn_unkeyed_model_handler >}}
-{{< /highlight >}}
-
-### Example 4: Sklearn keyed model
-
-In this example, we create a pipeline that uses an SKlearn RunInference
transform on keyed data.
-
-{{< highlight language="py"
file="sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
- class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference.py"
sklearn_keyed_model_handler >}}
-{{</ highlight >}}
-
-{{< paragraph class="notebook-skip" >}}
-Output:
-{{< /paragraph >}}
+The following examples show how to create pipelines that use the Beam
RunInference API to make predictions based on models.
-{{< highlight class="notebook-skip" >}}
-{{< code_sample
"sdks/python/apache_beam/examples/snippets/transforms/elementwise/runinference_test.py"
sklearn_keyed_model_handler >}}
-{{< /highlight >}}
+{{< table >}}
+| Framework | Example |
+| ----- | ----- |
+| PyTorch | [PyTorch unkeyed
model](/documentation/transforms/python/elementwise/runinference-pytorch/#example-1-pytorch-unkeyed-model)
|
+| PyTorch | [PyTorch keyed
model](/documentation/transforms/python/elementwise/runinference-pytorch/#example-2-pytorch-keyed-model)
|
+| Sklearn| [Sklearn unkeyed
model](/documentation/transforms/python/elementwise/runinference-sklearn/#example-1-sklearn-unkeyed-model)
|
+| Sklearn | [Sklearn keyed
model](/documentation/transforms/python/elementwise/runinference-sklearn/#example-2-sklearn-keyed-model)
|:
+{{< /table >}}
## Related transforms
diff --git
a/website/www/site/layouts/partials/section-menu/en/documentation.html
b/website/www/site/layouts/partials/section-menu/en/documentation.html
index 61d7aa9fe35..e10a76b9756 100755
--- a/website/www/site/layouts/partials/section-menu/en/documentation.html
+++ b/website/www/site/layouts/partials/section-menu/en/documentation.html
@@ -216,17 +216,50 @@
<span class="section-nav-list-title">AI/ML pipelines</span>
<ul class="section-nav-list">
- <li><a href="/documentation/ml/overview/">Overview</a></li>
- <li><a href="/documentation/ml/data-processing/">Data processing</a></li>
- <li><a href="/documentation/ml/runinference-metrics/">RunInference
Metrics</a></li>
- <li><a href="/documentation/ml/tensorrt-runinference">TensorRT
Inference</a></li>
- <li><a href="/documentation/ml/model-evaluation/">Model evaluation</a></li>
- <li><a href="/documentation/ml/orchestration/">Workflow
Orchestration</a></li>
- <li><a href="/documentation/ml/multi-model-pipelines/">Multi-model
pipelines</a></li>
- <li><a href="/documentation/ml/online-clustering/">Online
Clustering</a></li>
- <li><a href="/documentation/ml/anomaly-detection/">Anomaly
Detection</a></li>
- <li><a href="/documentation/ml/large-language-modeling">Large Language
Model Inference in Beam</a></li>
- <li><a href="/documentation/ml/per-entity-training">Per Entity Training in
Beam</a></li>
+ <li><a href="/documentation/ml/overview/">Get started with AI/ML</a></li>
+ <li><a href="/documentation/ml/about-ml/">About Beam ML</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Prediction and inference</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/ml/inference-overview/">Overview</a></li>
+ <li><a href="/documentation/ml/multi-model-pipelines/">Build a pipeline
with multiple models</a></li>
+ <li><a href="/documentation/ml/tensorrt-runinference">Build a custom
model handler with TensorRT</a></li>
+ <li><a href="/documentation/ml/large-language-modeling">Use LLM
inference</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Data processing</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/ml/data-processing/">Explore your
data</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Workflow orchestration</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/ml/orchestration/">Use ML-OPS workflow
orchestrators</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Model training</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/ml/per-entity-training">Per-entity
training</a></li>
+ <li><a href="/documentation/ml/online-clustering/">Online
clustering</a></li>
+ <li><a href="/documentation/ml/model-evaluation/">ML model
evaluation</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Use cases</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/ml/anomaly-detection/">Build an anomaly
detection pipeline</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Reference</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/ml/runinference-metrics/">RunInference
metrics</a></li>
+ <li><a href="/documentation/ml/model-evaluation/">Model
validation</a></li>
+ </ul>
+ </li>
</ul>
</li>
<li class="section-nav-item--collapsible">
@@ -260,7 +293,14 @@
<li><a
href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
<li><a
href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
<li><a
href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
- <li><a
href="/documentation/transforms/python/elementwise/runinference/">RunInference</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">RunInference</span>
+ <ul class="section-nav-list">
+ <li><a
href="/documentation/transforms/python/elementwise/runinference/">Overview</a></li>
+ <li><a
href="/documentation/transforms/python/elementwise/runinference-pytorch/">PyTorch
examples</a></li>
+ <li><a
href="/documentation/transforms/python/elementwise/runinference-sklearn/">Sklearn
examples</a></li>
+ </ul>
+ </li>
<li><a
href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
<li><a
href="/documentation/transforms/python/elementwise/values/">Values</a></li>
<li><a
href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
![https://beam.apache.org/images/logos/sdks/java.png"](https://beam.apache.org/images/logos/sdks/java.png"){kind=link}