jrmccluskey commented on code in PR #28160:
URL: https://github.com/apache/beam/pull/28160#discussion_r1305903150
##########
examples/notebooks/beam-ml/run_inference_vertex_ai.ipynb:
##########
@@ -0,0 +1,396 @@
+{
+ "cells": [
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "id": "fFjof1NgAJwu"
+ },
+ "outputs": [],
+ "source": [
+ "# @title ###### Licensed to the Apache Software Foundation (ASF),
Version 2.0 (the \"License\")\n",
+ "\n",
+ "# Licensed to the Apache Software Foundation (ASF) under one\n",
+ "# or more contributor license agreements. See the NOTICE file\n",
+ "# distributed with this work for additional information\n",
+ "# regarding copyright ownership. The ASF licenses this file\n",
+ "# to you under the Apache License, Version 2.0 (the\n",
+ "# \"License\"); you may not use this file except in compliance\n",
+ "# with the License. You may obtain a copy of the License at\n",
+ "#\n",
+ "# http://www.apache.org/licenses/LICENSE-2.0\n";,
+ "#\n",
+ "# Unless required by applicable law or agreed to in writing,\n",
+ "# software distributed under the License is distributed on an\n",
+ "# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "# KIND, either express or implied. See the License for the\n",
+ "# specific language governing permissions and limitations\n",
+ "# under the License"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "A8xNRyZMW1yK"
+ },
+ "source": [
+ "# Apache Beam RunInference with Vertex AI\n",
+ "\n",
+ "<table align=\"left\">\n",
+ " <td>\n",
+ " <a target=\"_blank\"
href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_vertex_ai.ipynb\";><img
src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\";
/>Run in Google Colab</a>\n",
+ " </td>\n",
+ " <td>\n",
+ " <a target=\"_blank\"
href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_vertex_ai.ipynb\";><img
src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\";
/>View source on GitHub</a>\n",
+ " </td>\n",
+ "</table>\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HrCtxslBGK8Z"
+ },
+ "source": [
+ "This notebook shows how to use the Apache Beam
[RunInference](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.RunInference)
transform for image classification with [Vertex
AI](https://cloud.google.com/vertex-ai).\n",
+ "Apache Beam has built-in support for sending requests to a remotely
deployed Vertex AI endpoint using the
[`VertexAIModelHandlerJSON`](https://github.com/apache/beam/blob/395c4d15bb74351b0aa020dc7463de8d85766e07/sdks/python/apache_beam/ml/inference/vertex_ai_inference.py#L61)
class. The input for this class is a JSON-serializable type, like a list.\n",
+ "\n",
+ "When you use remote inference with Vertex AI, consider the following
factors:\n",
+ "1. Public endpoints have a maximum request size of 1.5 MB. If you
want to send larger requests, you must configure a [private
endpoint](https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints)
and run your pipeline within the same VPC network. You might want to send
larger requests if you send batches of requests.\n",
+ "2. Inputs to the Vertex AI model handler must be JSON serializable.
If the inputs that aren't JSON serializable, the request to the endpoint
fails.\n",
+ "3. Hosting a model on Vertex AI and deploying it to an endpoint
incurs cost from Google Cloud.\n",
+ "\n",
+ "This notebook demonstrates the following steps:\n",
+ "- Configure access to a public Vertex AI endpoint.\n",
+ "- Set up example data.\n",
+ "- Run those examples with the built-in model handlers and get a
prediction inside an Apache Beam pipeline.\n",
+ "\n",
+ "For more information about using RunInference, see [Get started with
AI/ML pipelines](https://beam.apache.org/documentation/ml/overview/) in the
Apache Beam documentation."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "gVCtGOKTHMm4"
+ },
+ "source": [
+ "## Before you begin\n",
+ "Set up your environment and download dependencies."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ryC1AntVGtLi"
+ },
+ "source": [
+ "### Prerequisites\n",
+ "To run this notebook, first follow the steps for training a custom
model in the Vertex AI [\"Hello Custom
Training\"](https://cloud.google.com/vertex-ai/docs/tutorials/image-recognition-custom)
tutorial. At minimum, you need to have a trained image classification model
deployed to an endpoint within your Google Cloud project."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "YDHPlMjZRuY0"
+ },
+ "source": [
+ "### Install Apache Beam\n",
+ "To use RunInference with the built-in Vertex AI model handler,
install Apache Beam version 2.50.0 or later.\n",
+ "\n",
+ "**Note:** at time of writing 2.50.0 is in the release process and is
not available, so this notebook uses a release candidate build."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "jBakpNZnAhqk"
+ },
+ "outputs": [],
+ "source": [
+ "!pip install protobuf --quiet\n",
+ "!pip install apache_beam[gcp,interactive]==2.50.0rc1 --quiet\n",
+ "# Enforce shapely < 2.0.0 to avoid an issue with google.aiplatform\n",
+ "!pip install shapely==1.7.1 --quiet\n",
+ "\n",
+ "# To use the newly installed versions, restart the runtime.\n",
+ "exit()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "X80jy3FqHjK4"
+ },
+ "source": [
+ "### Authenticate with Google Cloud\n",
+ "This notebook relies on having a Vertex AI endpoint deployed to
Google Cloud. To use your Google Cloud account, authenticate this notebook."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "id": "Kz9sccyGBqz3"
+ },
+ "outputs": [],
+ "source": [
+ "from google.colab import auth\n",
+ "auth.authenticate_user()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "40qtP6zJuMXm"
+ },
+ "source": [
+ "### Import dependencies and set up your bucket\n",
+ "Use the following code to import dependencies and to set up your
Google Cloud Storage bucket.\n",
+ "\n",
+ "Replace `PROJECT_ID`, `LOCATION_NAME`, and `ENDPOINT_ID` with the ID
of your project, the GCP region where your model is deployed, and the ID of
your Vertex AI endpoint."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "O_a0-4Gb19cy"
+ },
+ "source": [
+ "### Query your Endpoint\n",
+ "\n",
+ "Verify that your model is deployed to your Vertex AI endpoint. If you
encounter errors, make sure that your endpoint is live and accessible from your
current account."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/";,
+ "height": 35
+ },
+ "id": "5XkIYXhJBFmS",
+ "outputId": "46b4c3ca-3428-465e-8c24-bfef525813fd"
+ },
+ "outputs": [
+ {
+ "data": {
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ },
+ "text/plain": [
+ "'hello_custom'"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "aiplatform.init(project=project, location=location)\n",
+ "endpoint = aiplatform.Endpoint(endpoint_name=endpoint_id)\n",
+ "# You can get more metadata if desired by removing
[0].display_name\n",
+ "endpoint.list_models()[0].display_name"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Y3BC2aY8cIMI"
+ },
+ "source": [
+ "### Preprocess an example image\n",
+ "\n",
+ "Preprocess an input to match what your model expects. Use the
following code to complete these steps:\n",
+ "\n",
+ "1. Test the model by using an image of sunflowers.\n",
Review Comment:
Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
![https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\"](https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\"){kind=link}
![https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\"](https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\"){kind=link}