charlespnh commented on issue #35788: URL: https://github.com/apache/beam/issues/35788#issuecomment-3156452390
> Started at the location below. > > [beam/sdks/python/apache_beam/yaml/yaml_provider.py](https://github.com/apache/beam/blob/720f9617be059492f9f4318d808f4a42ac20c9a6/sdks/python/apache_beam/yaml/yaml_provider.py#L439) > > Line 439 in [720f961](/apache/beam/commit/720f9617be059492f9f4318d808f4a42ac20c9a6) > > PypiExpansionService([ > Is there another error that indicate Python expansion service failing to start ? Full log message: ``` INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. Building pipeline... WARNING:root:Could not load ML transform module apache_beam.ml.transforms.embeddings.open_ai: No module named 'openai'. Please install the necessary module dependencies WARNING:root:Could not load ML transform module apache_beam.ml.transforms.embeddings.tensorflow_hub: No module named 'tensorflow'. Please install the necessary module dependencies INFO:apache_beam.yaml.yaml_transform:Expanding "ReadFromBigQuery" at line 23 WARNING:apache_beam.transforms.core:('No iterator is returned by the process method in %s.', <class 'apache_beam.io.gcp.bigquery_read_internal._PassThroughThenCleanupTempDatasets.expand.<locals>.CleanUpProjects'>) INFO:apache_beam.yaml.yaml_transform:Expanding "KNN" at line 29 INFO:apache_beam.utils.subprocess_server:Starting service with ['/Users/charlesnguyen/.apache_beam/cache/venvs/5316be40511ca369dcddc7cc8c711c62d053c3cdfda7fc4168fbb4232ba67964/bin/python' '-m' 'apache_beam.runners.portability.expansion_service_main' '--port' '62522' '--fully_qualified_name_glob=*' '--pickle_library=cloudpickle' '--requirements_file=/Users/charlesnguyen/.apache_beam/cache/venvs/5316be40511ca369dcddc7cc8c711c62d053c3cdfda7fc4168fbb4232ba67964-requirements.txt' '--serve_loopback_worker'] INFO:apache_beam.utils.subprocess_server:INFO:apache_beam.runners.portability.stager:Executing command: ['/Users/charlesnguyen/.apache_beam/cache/venvs/5316be40511ca369dcddc7cc8c711c62d053c3cdfda7fc4168fbb4232ba67964/bin/python', '-m', 'pip', 'download', '--dest', '/var/folders/c9/3ylgx85x553b8451_158qxpr0000gn/T/dataflow-requirements-cache', '-r', '/var/folders/c9/3ylgx85x553b8451_158qxpr0000gn/T/tmp_kdid1w4/tmp_requirements.txt', '--exists-action', 'i', '--no-deps', '--implementation', 'cp', '--abi', 'cp311', '--platform', 'manylinux2014_x86_64'] INFO:apache_beam.utils.subprocess_server:INFO:apache_beam.runners.portability.stager:Executing command: ['/Users/charlesnguyen/.apache_beam/cache/venvs/5316be40511ca369dcddc7cc8c711c62d053c3cdfda7fc4168fbb4232ba67964/bin/python', '-m', 'pip', 'download', '--dest', '/var/folders/c9/3ylgx85x553b8451_158qxpr0000gn/T/dataflow-requirements-cache', '-r', '/var/folders/c9/3ylgx85x553b8451_158qxpr0000gn/T/tmpc8_q_frj/tmp_requirements.txt', '--exists-action', 'i', '--no-deps', '--implementation', 'cp', '--abi', 'cp311', '--platform', 'manylinux2014_x86_64'] INFO:apache_beam.utils.subprocess_server:INFO:__main__:Listening for expansion requests at 62522 INFO:apache_beam.utils.subprocess_server:ERROR:grpc._server:Exception calling application: Method not implemented! INFO:apache_beam.utils.subprocess_server:Traceback (most recent call last): INFO:apache_beam.utils.subprocess_server: File "/Users/charlesnguyen/.apache_beam/cache/venvs/5316be40511ca369dcddc7cc8c711c62d053c3cdfda7fc4168fbb4232ba67964/lib/python3.11/site-packages/grpc/_server.py", line 610, in _call_behavior INFO:apache_beam.utils.subprocess_server: response_or_iterator = behavior(argument, context) INFO:apache_beam.utils.subprocess_server: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ INFO:apache_beam.utils.subprocess_server: File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/portability/api/org/apache/beam/model/job_management/v1/beam_expansion_api_pb2_grpc.py", line 46, in DiscoverSchemaTransform INFO:apache_beam.utils.subprocess_server: raise NotImplementedError('Method not implemented!') INFO:apache_beam.utils.subprocess_server:NotImplementedError: Method not implemented! INFO:apache_beam.utils.subprocess_server:INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. INFO:apache_beam.utils.subprocess_server:WARNING: All log messages before absl::InitializeLog() is called are written to STDERR INFO:apache_beam.utils.subprocess_server:I0000 00:00:1754420342.099952 8355449 fork_posix.cc:77] Other threads are currently calling into gRPC, skipping fork() handlers Traceback (most recent call last): File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 506, in expand_leaf_transform outputs = inputs | scope.unique_name(spec, ptransform) >> ptransform ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/pvalue.py", line 138, in __or__ return self.pipeline.apply(ptransform, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/pipeline.py", line 722, in apply return self.apply( ^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/pipeline.py", line 733, in apply return self.apply(transform, pvalueish) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/pipeline.py", line 813, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/runners/runner.py", line 191, in apply return self.apply_PTransform(transform, input, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/runners/runner.py", line 195, in apply_PTransform return transform.expand(input) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/transforms/external.py", line 824, in expand response = service.Expand(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/.virtualenvs/env/lib/python3.11/site-packages/grpc/_channel.py", line 1181, in __call__ return _end_unary_response_blocking(state, call, False, None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/.virtualenvs/env/lib/python3.11/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking raise _InactiveRpcError(state) # pytype: disable=not-instantiable ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "UNKNOWN:Error received from peer {grpc_message:"Socket closed", grpc_status:14, created_time:"2025-08-05T14:59:02.47977-04:00"}" > The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/main.py", line 297, in <module> run() File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/main.py", line 148, in run constructor(p) File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/main.py", line 281, in constructor yaml_transform.expand_pipeline( File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 1137, in expand_pipeline providers or {})).expand(root) ^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 1098, in expand result = expand_transform( ^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 478, in expand_transform return expand_composite_transform(spec, scope) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 561, in expand_composite_transform return transform.expand(None) ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 544, in expand inner_scope.compute_all() File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 205, in compute_all self.compute_outputs(transform_id) File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 103, in wrapper self._cache[key] = func(self, *args) ^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 241, in compute_outputs return expand_transform(self._transforms_by_uuid[transform_id], self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 480, in expand_transform return expand_leaf_transform(spec, scope) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/charlesnguyen/Code/beam/sdks/python/apache_beam/yaml/yaml_transform.py", line 508, in expand_leaf_transform raise ValueError( ValueError: Error applying transform "KNN" at line 29: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "UNKNOWN:Error received from peer {grpc_message:"Socket closed", grpc_status:14, created_time:"2025-08-05T14:59:02.47977-04:00"}" > ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
