Lee-W commented on code in PR #75369: URL: https://github.com/apache/airflow/pull/75369#discussion_r4520055186
########## providers/common/ai/docs/self_hosted_models.rst: ########## @@ -257,6 +257,59 @@ model doesn't have to guess a path. That matters because ``HttpHook`` raises on any response outside the 2xx/3xx range: if the model guessed wrong, the task would fail outright rather than the agent recovering gracefully. +Waiting for an Ollama model to load +------------------------------------- + +Ollama loads a model into memory on the first request, which can take long +enough to time out that request. To load the model ahead of time, call +:meth:`~airflow.providers.common.ai.hooks.pydantic_ai.PydanticAIHook.ensure_model_ready` +before the task builds its agent. It asks Ollama which models are loaded, +sends a load request if this connection's model is missing, waits for the load +to finish, then checks once that Ollama lists the model: + +.. code-block:: python + + from airflow.providers.common.ai.hooks.pydantic_ai import PydanticAIHook + + hook = PydanticAIHook(llm_conn_id="pydanticai_default") + hook.ensure_model_ready(timeout=300) + agent = hook.create_agent(instructions="Answer briefly.") + +Nothing calls it for you; it is opt-in. It has these limits: Review Comment: The guide now says the load request does not send `keep_alive`, so Ollama unloads the model after its default 5 minutes if the LLM task waits in the queue longer, and suggests `OLLAMA_KEEP_ALIVE` for that. `@task.llm` and `AgentOperator` users load the model from an upstream task. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
