[PR] [SEDONA-568] Refactor TestBaseScala to allow customizable Spark configurations [sedona]

[via GitHub]

zhangfengcdt opened a new pull request, #1455:
URL: https://github.com/apache/sedona/pull/1455

   - Added default Spark configurations in `TestBaseScala` trait.
   - Provided a method `defaultSparkConfig` for default configurations.
   - Allowed subclasses to override `sparkConfig` to add or modify Spark 
configurations.
   - Updated initialization of Spark session and Spark context to use the 
provided configurations.
   - Included comments to explain how to override configurations in subclasses.
   - Ensured that default configurations are preserved if not overridden by 
subclasses.
   
   
   
   ## Did you read the Contributor Guide?
   
   - Yes, I have read the [Contributor 
Rules](https://sedona.apache.org/latest-snapshot/community/rule/) and 
[Contributor Development 
Guide](https://sedona.apache.org/latest-snapshot/community/develop/)
   
   ## Is this PR related to a JIRA ticket?
   
   - Yes, the URL of the associated JIRA ticket is 
https://issues.apache.org/jira/browse/SEDONA-568. The PR name follows the 
format `[SEDONA-XXX] my subject`.
   
   ## What changes were proposed in this PR?
   - Added default Spark configurations in `TestBaseScala` trait.
   - Provided a method `defaultSparkConfig` for default configurations.
   - Allowed subclasses to override `sparkConfig` to add or modify Spark 
configurations.
   - Updated initialization of Spark session and Spark context to use the 
provided configurations.
   - Included comments to explain how to override configurations in subclasses.
   - Ensured that default configurations are preserved if not overridden by 
subclasses.
   
   After this change, user can either use 
   - Default Configurations: Provided in defaultSparkConfig.
   - Overriding Configurations: Subclasses can override sparkConfig and merge 
additional configurations with the default ones using the ++ operator, for 
example:
   
   ```
   class CustomTest extends TestBaseScala {
     // Override sparkConfig to provide additional configurations
     override def sparkConfig: Map[String, String] = defaultSparkConfig ++ Map(
       "spark.sql.warehouse.dir" -> "/custom/warehouse/location",
       "spark.some.config.option" -> "some-value"
     )
   
     describe("CustomTest") {
       it("should use custom Spark configuration") {
         assert(sparkSession.conf.get("spark.some.config.option") == 
"some-value")
       }
   
       it("should access Spark context") {
         assert(sc != null)
       }
     }
   }
   
   ```
   
   ## How was this patch tested?
   All existing unit tests should all pass without any impact.
   
   ## Did this PR include necessary documentation updates?
   
   - No, this PR does not affect any public API so no need to change the 
documentation.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sedona.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org