damccorm commented on code in PR #30351:
URL: https://github.com/apache/beam/pull/30351#discussion_r1519781906
##########
learning/prompts/documentation-lookup/42_ai_custom_inference.md:
##########
@@ -16,20 +16,23 @@ To execute external API calls with the `RunInference`
transform, you will need t
class CustomModelHandler(ModelHandler):
"""DoFn that accepts a batch of inputs and sends that batch to the remote
API for inference"""
- def load_model(self):
+ def initialize_client(self):
"""Initiate the Custom remote API client."""
client = ... # Initialize the client
return client
def run_inference(self, batch, model, inference):
+ # Initialize the client.
+ client = self.initialize_client()
+
# Prepare a batch request for all inputs in the batch.
inputs = ... # Process inputs from the batch
input_requests = ... # Prepare input requests for the model
batch_request = ... # Prepare batch request for the model
# Send the batch request to the remote endpoint.
- responses = model.(request=batch_request).responses
+ responses = client.(request=batch_request).responses
Review Comment:
```suggestion
responses = client.request(request=batch_request).responses
```
##########
learning/prompts/documentation-lookup-nolinks/42_ai_custom_inference.md:
##########
@@ -16,20 +16,23 @@ To execute external API calls with the `RunInference`
transform, you will need t
class CustomModelHandler(ModelHandler):
"""DoFn that accepts a batch of inputs and sends that batch to the remote
API for inference"""
- def load_model(self):
+ def initialize_client(self):
"""Initiate the Custom remote API client."""
client = ... # Initialize the client
return client
def run_inference(self, batch, model, inference):
+ # Initialize the client.
+ client = self.initialize_client()
+
# Prepare a batch request for all inputs in the batch.
inputs = ... # Process inputs from the batch
input_requests = ... # Prepare input requests for the model
batch_request = ... # Prepare batch request for the model
# Send the batch request to the remote endpoint.
- responses = model.(request=batch_request).responses
+ responses = client.(request=batch_request).responses
Review Comment:
```suggestion
responses = client.request(request=batch_request).responses
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
