Raja10D opened a new issue, #4835: URL: https://github.com/apache/hop/issues/4835
### What needs to happen? I am writing to report an issue we are encountering when running a Hop pipeline on a Spark Standalone cluster. Our Spark-Submit command works perfectly in a local environment with the following configuration: **spark-submit \ --master local[4] \ --class org.apache.hop.beam.run.MainBeam \ --driver-java-options '-DPROJECT_HOME=/home/decoders/hop_test_2_8_0_backup/hop/config/projects/default' \ /home/decoders/hop_test_2_8_0_backup/hop/fat-jar.jar \ /home/decoders/hop_test_2_8_0_backup/hop/Creating_5_payoutAmt_source_Dec23.hpl \ /home/decoders/hop_test_2_8_0_backup/hop/metadata.json \ beam_runner_spark** However, when we replace local[4] with the Spark Standalone cluster URL (spark://10d154:7077), the pipeline fails to run in our real-time production environment. We have verified that all necessary dependencies are included in the fat-jar.jar and reviewed the configuration settings, but the issue persists. Could you please provide guidance on any additional files or settings that might be required to run the Hop pipeline successfully on a Spark Standalone cluster? Thank you for your assistance. ### Issue Priority Priority: 1 ### Issue Component Component: Other, Component: Hop Run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
