yaooqinn opened a new pull request, #49617: URL: https://github.com/apache/spark/pull/49617
### What changes were proposed in this pull request? This PR makes the SparkPi Scala example able to run against Spark Connect Servers by replacing the usage of SparkConext/RDD APIs with SparkSession/DataFrame APIs. ### Why are the changes needed? The SparkPi example, mostly as the first step for users to get to know spark, should be able to run on Spark Connect mode. ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? Manually build and test ```log bin/spark-submit --remote 'sc://localhost' --class org.apache.spark.examples.SparkPi examples/jars/spark-examples_2.13-4.1.0-SNAPSHOT.jar WARNING: Using incubator modules: jdk.incubator.vector 25/01/23 15:00:03 INFO BaseAllocator: Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator=true. 25/01/23 15:00:03 INFO DefaultAllocationManagerOption: allocation manager type not specified, using netty as the default type 25/01/23 15:00:03 INFO CheckAllocator: Using DefaultAllocationManager at memory/netty/DefaultAllocationManagerFactory.class Pi is roughly 3.1388756943784717 25/01/23 15:00:04 INFO ShutdownHookManager: Shutdown hook called 25/01/23 15:00:04 INFO ShutdownHookManager: Deleting directory /private/var/folders/84/dgr9ykwn6yndcmq1kjxqvk200000gn/T/spark-25ed842e-5888-47ce-bb0b-442385d643cb ``` ### Was this patch authored or co-authored using generative AI tooling? <!-- If generative AI tooling has been used in the process of authoring this patch, please include the phrase: 'Generated-by: ' followed by the name of the tool and its version. If no, write 'No'. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details. --> no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
