Your original log messages include error messages in this file - https://github.com/apache/beam/blob/master/sdks/python/container/boot.go#L253 .
It may be due to failure of the Python harness process. On Tue, 28 Jan 2025 at 06:22, Lydian Lee <tingyenlee...@gmail.com> wrote: > Hi, > > It seems like there's also similar issue from others: > https://github.com/apache/beam/issues/29683 > > For me, here's how I launch the job with the following argos: > --runner=portableRunner > --streaming > --environment_type=EXTERNAL > --environment_config=localhost:50000 > --experiments=disable_logging_submission_environment # without this, it > is complaining on rpc error: code = Unknown desc = ; failed to retrieve > chunk for /tmp/staged/submission_environment_dependencies.txt > > For launching the harness: > /opt/apache/beam/boot --worker_pool > > > I've tested the different pipeline which all works in 2.41.0, but now they > all failed with 2.60.0. I've also tested with 2.57.0, and that one seems to > already have the same issue. > > Something worth mentioning is I also need to install > `google-cloud-storage` when bumping the version, otherwise it will complain > about missing the google cloud library, although I am not using gcs at all. > ``` > Starting worker with command ['/opt/apache/beam/boot', '--id=31-1', > '--logging_endpoint=localhost:40649', > '--artifact_endpoint=localhost:36751', > '--provision_endpoint=localhost:45679', '--control_endpoint=localh 2 > 025/01/23 23:46:14 Provision info: > pipeline_options:{fields:{key:"beam:option:allow_non_deterministic_key_coders:v1" > value:{bool_value:false}} > fields:{key:"beam:option:allow_non_restored_state:v1" > value:{bool_value:false}} fields:{key:"beam:o 2 > 025/01/23 23:46:28 boot.go: error logging message over FnAPI. endpoint > localhost:40649 error: EOF message follows > 2025/01/23 23:46:28 DEBUG Received signal: terminated > 2025/01/23 23:46:28 boot.go: error logging message over FnAPI. endpoint > localhost:40649 error: EOF message follows > 2025/01/23 23:46:28 ERROR Failed to import GCSFileSystem; loading of this > filesystem will be skipped. Error details: cannot import name 'storage' > from 'google.cloud' (unknown location) > ``` > > Thanks for your help. >
