[ 
https://issues.apache.org/jira/browse/SPARK-61788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruifeng Zheng updated SPARK-61788:
----------------------------------
    Affects Version/s: 4.5.0

> Fix StateStoreInstanceMetricSuite depending on the state store maintenance 
> task of an earlier test
> --------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61788
>                 URL: https://issues.apache.org/jira/browse/SPARK-61788
>             Project: Spark
>          Issue Type: Test
>          Components: Structured Streaming, Tests
>    Affects Versions: 5.0.0, 4.5.0
>            Reporter: Raviston Thanasekar
>            Assignee: Raviston Thanasekar
>            Priority: Major
>              Labels: pull-request-available
>
> StateStoreInstanceMetricSuiteWithRowChecksum "SPARK-51097: Verify snapshot 
> lag metrics are updated correctly with RocksDBStateStoreProvider (with 
> changelog checkpointing)" fails when another test in the same JVM has started 
> the state store maintenance task before it:
> {code}
> Assert on query failed: Execute: The code passed to eventually never returned 
> normally. Attempted 669 times over 10.005817232000002 seconds. Last failure 
> message: instanceMetrics.forall(((x$2: (String, Long)) => 
> scala.Predef.Long2long(x$2._2).>=(0))) was false.
> {code}
> The maintenance task is shared by the JVM, and 
> StateStore.startMaintenanceIfNeeded only starts one when none is running. So 
> it keeps the interval of the test that started it. The test sets 
> spark.sql.streaming.stateStore.maintenanceInterval to 100 ms, but when a task 
> with the default interval of 60 seconds is already running, no snapshot is 
> uploaded during the test, and every SnapshotLastUploaded metric stays -1.
> In "sql - other tests", WriteDistributionAndOrderingSuite ("ordered 
> distribution and sort with same exprs: micro-batch complete") started the 
> task just before this test, e.g. in 
> https://github.com/raviston/spark/actions/runs/43067912461 (attempts 10 to 
> 12). StateStoreInstanceMetricSuiteWithRowChecksum runs in that group because 
> the ExtendedSQLTest tag of StateStoreInstanceMetricSuite is not inherited.
> To reproduce:
> {code}
> build/sbt 'sql/testOnly 
> org.apache.spark.sql.connector.WriteDistributionAndOrderingSuite 
> org.apache.spark.sql.execution.streaming.state.StateStoreInstanceMetricSuiteWithRowChecksum
>  -- -z "ordered distribution and sort with same exprs: micro-batch complete" 
> -z "SPARK-51097: Verify snapshot lag metrics are updated correctly with 
> RocksDBStateStoreProvider (with changelog"'
> {code}



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to