hi Sofia,
here you have nice example
https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/custom_remote_inference.ipynb
where CloudVisionModelHandler is custom code that can invoke any client
library.
you can pass the key as one of the constructors to CloudVisionModelHandler
or you can load it from any preferred secret managers e.g.
def load_model(self):
"""Initiate the OAI API client."""
client = OpenAI(
api_key=client.access_secret_version(name="OPENAI_API_KEY").data.decode('UTF-8')
)
return client
def run_inference(self, batch, oai_client, inference):
response = oai_client.responses.create( # your LLM magic goes here.
[..]
using env variables won't work here as like you've noticed, beam is running
on multiple machines and you can't set env variables there.
On Thu, Mar 27, 2025 at 9:22 AM Sofia’s World <mmistr...@gmail.com> wrote:
> Hello
> presumably is possible to kick off a beam process that invokes an LLM>?
> but the only issue i have is how/where do i store the OpenAI key for
> example
>
> in my c current colab/pc setup i have the key configured in my
> environment.. but Beam will run
> on multiple machines... how do i configure the OPENAI_KEY for example?
>
> thanks and regards
> Marco
>
