riteshghorse commented on code in PR #28474:
URL: https://github.com/apache/beam/pull/28474#discussion_r1330893673
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -593,9 +594,18 @@ def __init__(
Args:
task (str or enum.Enum): task supported by HuggingFace Pipelines.
Accepts a string task or an enum.Enum from PipelineTask.
- model : path to pretrained model on Hugging Face Models Hub to use custom
- model for the chosen task. If the model already defines the task then
- no need to specify the task parameter.
+ model (str): path to the pretrained *model-id* on Hugging Face Models Hub
Review Comment:
it is like defining the model with config first and then passing it like
this:
```
model_id = 'meta-llama/Llama-2-7b-hf'
model_config = transformers.AutoConfig.from_pretrained(
model_id,
)
model = transformers.AutoModelForCausalLM.from_pretrained(
model_id,
config=model_config,
)
model_handler = HuggingFacePipelineModelHandler(
task='text-generation',
model = model,
load_pipeline_args = {'framework' : 'pt'},
inference_args = {'max_length' : 200}
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
