damccorm commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1281062630
##########
sdks/python/apache_beam/ml/inference/huggingface_inference_it_test.py:
##########
@@ -74,6 +75,37 @@ def test_hf_language_modeling(self):
predicted_predicted_text = predictions_dict[text]
self.assertEqual(actual_predicted_text, predicted_predicted_text)
+ def test_hf_pipeline(self):
+ test_pipeline = TestPipeline(is_integration_test=True)
+ # Path to text file containing some questions and context
+ input_file = 'gs://apache-beam-ml/datasets/custom/questions.txt'
+ output_file_dir = 'gs://apache-beam-ml/hf/testing/predictions'
Review Comment:
I actually mean the model itself that the pipeline is built on top of, e.g.
if they update from v2 to v3 -
https://huggingface.co/deepset/roberta-base-squad2/commit/deedc3e42208524e0df3d9149d1f26aa6934f05f
I guess that's _that_ likely to happen, but we could just pin to v2 or even
a commit sha with the revision parameter -
https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.pipeline.revision
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
