TheNeuralBit opened a new issue, #22413: URL: https://github.com/apache/beam/issues/22413
### What happened? `:torchInferenceTest` is broken in Python 3.9 and 3.7 PostCommits. Interesting 3.8 seems to be ok. I dug into one of the failures (https://ci-beam.apache.org/job/beam_PostCommit_Python39/635/) and found the following error log: ``` 05:20:28 > Task :sdks:python:test-suites:direct:py39:torchInferenceTest 05:20:28 05:20:28 [gw1] PASSED apache_beam/ml/inference/pytorch_inference_it_test.py::PyTorchInference::test_torch_run_inference_imagenet_mobilenetv2 05:20:41 [gw0] PASSED apache_beam/ml/inference/pytorch_inference_it_test.py::PyTorchInference::test_torch_run_inference_bert_for_masked_lm 05:20:43 [gw2] FAILED apache_beam/ml/inference/pytorch_inference_it_test.py::PyTorchInference::test_torch_run_inference_coco_maskrcnn_resnet50_fpn 05:20:43 05:20:43 =================================== FAILURES =================================== 05:20:43 _____ PyTorchInference.test_torch_run_inference_coco_maskrcnn_resnet50_fpn _____ 05:20:43 [gw2] linux -- Python 3.9.10 /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python39/src/build/gradleenv/1398941893/bin/python3.9 05:20:43 05:20:43 self = <apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference testMethod=test_torch_run_inference_coco_maskrcnn_resnet50_fpn> 05:20:43 05:20:43 @pytest.mark.uses_pytorch 05:20:43 @pytest.mark.it_postcommit 05:20:43 def test_torch_run_inference_coco_maskrcnn_resnet50_fpn(self): 05:20:43 test_pipeline = TestPipeline(is_integration_test=True) 05:20:43 # text files containing absolute path to the coco validation data on GCS 05:20:43 file_of_image_names = 'gs://apache-beam-ml/testing/inputs/it_coco_validation_inputs.txt' # pylint: disable=line-too-long 05:20:43 output_file_dir = 'gs://apache-beam-ml/testing/predictions' 05:20:43 output_file = '/'.join([output_file_dir, str(uuid.uuid4()), 'result.txt']) 05:20:43 05:20:43 model_state_dict_path = 'gs://apache-beam-ml/models/torchvision.models.detection.maskrcnn_resnet50_fpn.pth' # pylint: disable=line-too-long 05:20:43 images_dir = 'gs://apache-beam-ml/datasets/coco/raw-data/val2017' 05:20:43 extra_opts = { 05:20:43 'input': file_of_image_names, 05:20:43 'output': output_file, 05:20:43 'model_state_dict_path': model_state_dict_path, 05:20:43 'images_dir': images_dir, 05:20:43 } 05:20:43 pytorch_image_segmentation.run( 05:20:43 test_pipeline.get_full_options_as_args(**extra_opts), 05:20:43 save_main_session=False) 05:20:43 05:20:43 self.assertEqual(FileSystems().exists(output_file), True) 05:20:43 predictions = process_outputs(filepath=output_file) 05:20:43 actuals_file = 'gs://apache-beam-ml/testing/expected_outputs/test_torch_run_inference_coco_maskrcnn_resnet50_fpn_actuals.txt' # pylint: disable=line-too-long 05:20:43 actuals = process_outputs(filepath=actuals_file) 05:20:43 05:20:43 predictions_dict = {} 05:20:43 for prediction in predictions: 05:20:43 filename, prediction_labels = prediction.split(';') 05:20:43 predictions_dict[filename] = prediction_labels 05:20:43 05:20:43 for actual in actuals: 05:20:43 filename, actual_labels = actual.split(';') 05:20:43 prediction_labels = predictions_dict[filename] 05:20:43 > self.assertEqual(actual_labels, prediction_labels) 05:20:43 E AssertionError: "['refrigerator', 'oven', 'orange', 'dining [534 chars]or']" != "['traffic light', 'car', 'traffic light', '[15 chars]ar']" 05:20:43 E Diff is 650 characters long. Set self.maxDiff to None to see it. 05:20:43 05:20:43 apache_beam/ml/inference/pytorch_inference_it_test.py:128: AssertionError ``` ### Issue Priority Priority: 1 ### Issue Component Component: sdk-py-core -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
