ericm-db commented on code in PR #56015:
URL: https://github.com/apache/spark/pull/56015#discussion_r3283111281
##########
sql/core/src/test/scala/org/apache/spark/sql/streaming/test/StreamingSourceEvolutionSuite.scala:
##########
@@ -506,13 +534,12 @@ class StreamingSourceEvolutionSuite extends StreamTest {
/**
* Helper method to run tests with source evolution enabled.
- * Sets offset log format to V2 (OffsetMap) since named sources require it.
+ * Enabling source evolution automatically forces offset log format V2
(OffsetMap) for new
+ * queries, since named sources require it.
*/
def testWithSourceEvolution(testName: String, testTags: Tag*)(testBody: =>
Any): Unit = {
test(testName, testTags: _*) {
- withSQLConf(
- SQLConf.ENABLE_STREAMING_SOURCE_EVOLUTION.key -> "true",
- SQLConf.STREAMING_OFFSET_LOG_FORMAT_VERSION.key -> "2") {
+ withSQLConf(SQLConf.ENABLE_STREAMING_SOURCE_EVOLUTION.key -> "true") {
Review Comment:
Good catch — `ClientStreamingQuerySuite` had the same helper setting both
configs. Updated in 2864ae64477 to drop the explicit
`offsetLog.formatVersion=2`, since enabling source evolution now forces V2
automatically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
