v-kkhuang opened a new pull request, #5398: URL: https://github.com/apache/linkis/pull/5398
<!-- Thank you for sending the PR! We appreciate you spending the time to work on these changes. You can learn more about contributing to Apache Linkis here: https://linkis.apache.org/community/how-to-contribute Happy contributing! --> ### What is the purpose of the change **Background/Problem:** Currently, the Spark engine in Linkis does not fully support passing driver-specific parameters to the driver side. Users need to configure driver parameters through complex workarounds or manual configuration files, which reduces usability. **Purpose of Change:** To address this problem, this PR adds support for setting driver parameters directly to the driver side in the Spark engine, allowing users to configure driver-specific settings more conveniently through Linkis configuration. **Value/Impact:** After the change, users can pass driver parameters such as memory, cores, or Java options directly through Linkis configuration, improving flexibility and ease of use for Spark task submissions. ### Related issues/PRs Related issues: close #5397 Related pr:none ### Brief change log - Add driver parameter configuration support to SparkConfiguration - Update SparkEngineConnExecutor to pass driver parameters correctly to driver side - Add configuration tests for spark executor parameters - Add documentation including design, requirements, and test cases ### Checklist - [x] I have read the [Contributing Guidelines on pull requests](https://linkis.apache.org/community/how-to-contribute). - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [ ] I have added tests corresponding to this change - [ ] I have updated the documentation to reflect this change - [x] I have verified that this change is backward compatible - [ ] **If this is a code change**: I have written unit tests to fully verify the new behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
