This is an automated email from the ASF dual-hosted git repository.
gengliangwang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f7f32dbe1c78 [SPARK-58909][SQL][TEST] Move heavy suites out of the
sql-other GHA shard
f7f32dbe1c78 is described below
commit f7f32dbe1c78ca499c1dc2eae53f6a532f50bc3e
Author: Serge Rielau <[email protected]>
AuthorDate: Thu Aug 20 14:04:11 2026 -0700
[SPARK-58909][SQL][TEST] Move heavy suites out of the sql-other GHA shard
### What changes were proposed in this pull request?
Rebalance the GitHub Actions `sql - other tests` shard by tagging the
slowest untagged SQL suites so they run in `sql - extended tests` or `sql -
slow tests`.
`SlowSQLTest` (streaming family, same destination as SPARK-53068):
- `TransformWithStateInitialStateSuiteCheckpointV2` (parent was already
tagged; this subclass was not)
- `RealTimeTransformWithStateSuite` and
`RealTimeTransformWithStateSuiteWithRowChecksum`
- `StatePartitionAllColumnFamiliesWriterSuite`
- `StreamingSourceEvolutionSuite`
`ExtendedSQLTest`:
- `AdaptivePartialAggregationSuite`, `SortSuite`
- Default collation V1/V2 STRING and CHAR/VARCHAR suites
- `CollationSuite`, `ParquetGeoSuite`, `ParquetTypeWideningSuite`
- `OrcV1QuerySuite` / `OrcV2QuerySuite`, `CSVv1Suite` / `CSVv2Suite`
- `InsertSuite` (`sql/core`), `KeyGroupedPartitioningSuite`,
`SQLLastAttemptMetricPlanShapesSuite`
- `RocksDBDecoupledMaintenanceSuite` and
`RocksDBDecoupledMaintenanceSuiteWithRowChecksum`
No workflow or timeout changes.
### Why are the changes needed?
`sql - other tests` is running against the 150-minute `build` job timeout.
On GitHub-hosted runners it recently finished in about 141 minutes
(https://github.com/srielau/spark/actions/runs/32331062736), so a slightly
slower runner times the job out with no useful test report.
The other SQL shards still have spare capacity (`sql - extended tests` ~56
minutes, `sql - slow tests` ~105 minutes on the same run). Moving these suites
is the same approach as SPARK-44034 / SPARK-53068 / SPARK-54460.
Suite list came from a successful `sql - other` log, using each suite's
ScalaTest start banner and thread-leak end marker. The tagged set is about
35-40 minutes of wall time.
JIRA: https://issues.apache.org/jira/browse/SPARK-58909
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Test-only annotations; no production code change. Existing suites are
unchanged besides the CI tag. GitHub Actions on this PR will show whether `sql
- other`, `sql - slow`, and `sql - extended` all finish under the 150-minute
cap.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor Grok 4.6
Closes #58169 from srielau/SPARK-58909-rebalance-sql-other.
Authored-by: Serge Rielau <[email protected]>
Signed-off-by: Gengliang Wang <[email protected]>
---
.../src/test/scala/org/apache/spark/sql/MetadataCacheSuite.scala | 2 ++
.../src/test/scala/org/apache/spark/sql/NestedDataSourceSuite.scala | 2 ++
.../spark/sql/collation/CollatedFilterPushDownToParquetSuite.scala | 2 ++
.../test/scala/org/apache/spark/sql/collation/CollationSuite.scala | 2 ++
.../org/apache/spark/sql/collation/DefaultCollationTestSuite.scala | 5 +++++
.../org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala | 2 ++
.../src/test/scala/org/apache/spark/sql/execution/SortSuite.scala | 2 ++
.../sql/execution/aggregate/AdaptivePartialAggregationSuite.scala | 2 ++
.../spark/sql/execution/datasources/PushVariantIntoScanSuite.scala | 2 ++
.../org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala | 2 ++
.../execution/datasources/json/ExplodeEmbeddedArrayJsonSuite.scala | 2 ++
.../org/apache/spark/sql/execution/datasources/json/JsonSuite.scala | 2 ++
.../apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala | 2 ++
.../apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala | 2 ++
.../sql/execution/datasources/parquet/ParquetFileFormatSuite.scala | 2 ++
.../spark/sql/execution/datasources/parquet/ParquetGeoSuite.scala | 2 ++
.../spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala | 2 ++
.../sql/execution/datasources/parquet/ParquetTypeWideningSuite.scala | 2 ++
.../org/apache/spark/sql/execution/datasources/text/TextSuite.scala | 2 ++
.../spark/sql/execution/datasources/text/WholeTextFileSuite.scala | 2 ++
.../sql/execution/metric/SQLLastAttemptMetricPlanShapesSuite.scala | 2 ++
.../streaming/state/StatePartitionAllColumnFamiliesWriterSuite.scala | 2 ++
.../streaming/state/StateStoreDecoupledMaintenanceSuite.scala | 2 ++
.../src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala | 2 ++
.../apache/spark/sql/streaming/RealTimeTransformWithStateSuite.scala | 3 +++
.../spark/sql/streaming/TransformWithStateInitialStateSuite.scala | 1 +
.../spark/sql/streaming/test/StreamingSourceEvolutionSuite.scala | 2 ++
27 files changed, 57 insertions(+)
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/MetadataCacheSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/MetadataCacheSuite.scala
index e9d272728c94..a8b68ab77300 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/MetadataCacheSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/MetadataCacheSuite.scala
@@ -22,6 +22,7 @@ import java.io.File
import org.apache.spark.{SparkConf, SparkException}
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
+import org.apache.spark.tags.ExtendedSQLTest
/**
* Test suite to handle metadata cache related.
@@ -129,6 +130,7 @@ class MetadataCacheV1Suite extends MetadataCacheSuite {
}
}
+@ExtendedSQLTest
class MetadataCacheV2Suite extends MetadataCacheSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/NestedDataSourceSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/NestedDataSourceSuite.scala
index 4f2794ff87dc..017fe22c666e 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/NestedDataSourceSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/NestedDataSourceSuite.scala
@@ -20,6 +20,7 @@ import org.apache.spark.SparkConf
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types.{LongType, StructType}
+import org.apache.spark.tags.ExtendedSQLTest
// Datasource tests for nested schemas
trait NestedDataSourceSuiteBase extends SharedSparkSession {
@@ -83,6 +84,7 @@ class NestedDataSourceV1Suite extends
NestedDataSourceSuiteBase {
.set(SQLConf.USE_V1_SOURCE_LIST, nestedDataSources.mkString(","))
}
+@ExtendedSQLTest
class NestedDataSourceV2Suite extends NestedDataSourceSuiteBase {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/collation/CollatedFilterPushDownToParquetSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/collation/CollatedFilterPushDownToParquetSuite.scala
index 19e60f6c6276..bdbbf8057cf1 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/collation/CollatedFilterPushDownToParquetSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/collation/CollatedFilterPushDownToParquetSuite.scala
@@ -31,6 +31,7 @@ import
org.apache.spark.sql.execution.datasources.v2.parquet.ParquetScan
import org.apache.spark.sql.internal.{LegacyBehaviorPolicy, SQLConf}
import org.apache.spark.sql.sources.{EqualTo, Filter, IsNotNull}
import org.apache.spark.sql.test.SharedSparkSession
+import org.apache.spark.tags.ExtendedSQLTest
abstract class CollatedFilterPushDownToParquetSuite extends SharedSparkSession
with AdaptiveSparkPlanHelper {
@@ -248,6 +249,7 @@ class CollatedFilterPushDownToParquetV1Suite extends
CollatedFilterPushDownToPar
}
}
+@ExtendedSQLTest
class CollatedFilterPushDownToParquetV2Suite extends
CollatedFilterPushDownToParquetSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/collation/CollationSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/collation/CollationSuite.scala
index 045748ad7cb1..76a28e1af51e 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/collation/CollationSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/collation/CollationSuite.scala
@@ -41,7 +41,9 @@ import org.apache.spark.sql.execution.joins._
import org.apache.spark.sql.functions.col
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.types.{ArrayType, IntegerType, MapType, Metadata,
MetadataBuilder, StringType, StructField, StructType}
+import org.apache.spark.tags.ExtendedSQLTest
+@ExtendedSQLTest
class CollationSuite extends DatasourceV2SQLBase with AdaptiveSparkPlanHelper {
protected val v2Source = classOf[FakeV2ProviderWithCustomSchema].getName
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/collation/DefaultCollationTestSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/collation/DefaultCollationTestSuite.scala
index 82bb61648023..1c0c0c965dac 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/collation/DefaultCollationTestSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/collation/DefaultCollationTestSuite.scala
@@ -27,6 +27,7 @@ import
org.apache.spark.sql.connector.catalog.CatalogManager.SESSION_CATALOG_NAM
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types.{BooleanType, StringType, StructType}
+import org.apache.spark.tags.ExtendedSQLTest
abstract class DefaultCollationTestSuite extends SharedSparkSession {
@@ -1935,6 +1936,7 @@ abstract class DefaultCollationTestSuiteV2
}
}
+@ExtendedSQLTest
class DefaultCollationStringTestSuiteV1 extends DefaultCollationTestSuiteV1 {
override protected def testDataType(testName: String)(testFn: String =>
Unit): Unit = {
test(s"$testName [STRING]") {
@@ -1943,6 +1945,7 @@ class DefaultCollationStringTestSuiteV1 extends
DefaultCollationTestSuiteV1 {
}
}
+@ExtendedSQLTest
class DefaultCollationStringTestSuiteV2 extends DefaultCollationTestSuiteV2 {
override protected def testDataType(testName: String)(testFn: String =>
Unit): Unit = {
test(s"$testName [STRING]") {
@@ -1974,6 +1977,7 @@ class DefaultCollationStringTestSuiteV2 extends
DefaultCollationTestSuiteV2 {
}
}
+@ExtendedSQLTest
class DefaultCollationCharVarcharTestSuiteV1 extends
DefaultCollationTestSuiteV1 {
override protected def excluded: Seq[String] =
super.excluded ++ stringTestNames ++ stringTestNamesV1
@@ -1988,6 +1992,7 @@ class DefaultCollationCharVarcharTestSuiteV1 extends
DefaultCollationTestSuiteV1
}
}
+@ExtendedSQLTest
class DefaultCollationCharVarcharTestSuiteV2 extends
DefaultCollationTestSuiteV2 {
override protected def excluded: Seq[String] =
super.excluded ++ stringTestNames
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
index a732294786b2..e400d890fa50 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
@@ -45,7 +45,9 @@ import org.apache.spark.sql.functions.{col, max}
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.internal.SQLConf._
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
+@ExtendedSQLTest
class KeyGroupedPartitioningSuite extends DistributionAndOrderingSuiteBase
with ExplainSuiteHelper {
private val functions = Seq(
UnboundYearsFunction,
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/SortSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/SortSuite.scala
index f6493a8d0e85..7405aa5fd4de 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/SortSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/SortSuite.scala
@@ -25,11 +25,13 @@ import org.apache.spark.sql.catalyst.dsl.expressions._
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
/**
* Test sorting. Many of the test cases generate random data and compares the
sorted result with one
* sorted by a reference implementation ([[ReferenceSort]]).
*/
+@ExtendedSQLTest
class SortSuite extends SharedSparkSession {
import testImplicits.newProductEncoder
import testImplicits.localSeqToDatasetHolder
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/aggregate/AdaptivePartialAggregationSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/aggregate/AdaptivePartialAggregationSuite.scala
index 1727a3415a5b..baafe42e4058 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/aggregate/AdaptivePartialAggregationSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/aggregate/AdaptivePartialAggregationSuite.scala
@@ -23,6 +23,7 @@ import
org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper
import org.apache.spark.sql.functions._
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
+import org.apache.spark.tags.ExtendedSQLTest
/**
* Tests for runtime adaptive partial aggregation
@@ -43,6 +44,7 @@ import org.apache.spark.sql.test.SharedSparkSession
* when) it should -- high-cardinality input bypasses, low-cardinality
input keeps aggregating,
* the feature switch and eligibility rules are honored, and both check
points work.
*/
+@ExtendedSQLTest
class AdaptivePartialAggregationSuite extends QueryTest with SharedSparkSession
with AdaptiveSparkPlanHelper {
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PushVariantIntoScanSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PushVariantIntoScanSuite.scala
index e92e938ad142..73590b039a01 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PushVariantIntoScanSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PushVariantIntoScanSuite.scala
@@ -26,6 +26,7 @@ import
org.apache.spark.sql.execution.datasources.v2.DataSourceV2ScanRelation
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
trait PushVariantIntoScanSuiteBase extends SharedSparkSession {
override def sparkConf: SparkConf =
@@ -2719,6 +2720,7 @@ abstract class PushVariantIntoScanV2SuiteBase extends
QueryTest with PushVariant
}
// V2 DataSource tests - Row-based reader
+@ExtendedSQLTest
class PushVariantIntoScanV2Suite extends PushVariantIntoScanV2SuiteBase {
override protected def vectorizedReaderEnabled: Boolean = false
override protected def readerName: String = "row-based reader"
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
index 71eb34134920..3aefe25f6a08 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
@@ -48,6 +48,7 @@ import org.apache.spark.sql.internal.{LegacyBehaviorPolicy,
SQLConf}
import org.apache.spark.sql.internal.SQLConf.BinaryOutputStyle
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
abstract class CSVSuite
extends SharedSparkSession
@@ -4104,6 +4105,7 @@ class CSVv1Suite extends CSVSuite {
}
}
+@ExtendedSQLTest
class CSVv2Suite extends CSVSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/ExplodeEmbeddedArrayJsonSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/ExplodeEmbeddedArrayJsonSuite.scala
index 075f61042d90..a21d01af1d31 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/ExplodeEmbeddedArrayJsonSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/ExplodeEmbeddedArrayJsonSuite.scala
@@ -27,6 +27,7 @@ import org.apache.spark.sql.{AnalysisException, QueryTest,
Row}
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types.StructType
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.Utils
class EmbeddedArraySplitterSuite extends SparkFunSuite {
@@ -388,6 +389,7 @@ class ExplodeEmbeddedArrayJsonV1Suite extends
ExplodeEmbeddedArrayJsonSuite {
.set(SQLConf.USE_V1_SOURCE_LIST, "json")
}
+@ExtendedSQLTest
class ExplodeEmbeddedArrayJsonV2Suite extends ExplodeEmbeddedArrayJsonSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
index 308b41d9ec77..4cd6783e1df6 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
@@ -53,6 +53,7 @@ import org.apache.spark.sql.types._
import org.apache.spark.sql.types.StructType.fromDDL
import org.apache.spark.sql.types.TestUDT.{MyDenseVector, MyDenseVectorUDT}
import org.apache.spark.sql.util.CaseInsensitiveStringMap
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.ArrayImplicits._
import org.apache.spark.util.Utils
@@ -4356,6 +4357,7 @@ class JsonV1Suite extends JsonSuite {
.set(SQLConf.USE_V1_SOURCE_LIST, "json")
}
+@ExtendedSQLTest
class JsonV2Suite extends JsonSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala
index b2821c3d5fad..de72dd81e7e4 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala
@@ -44,6 +44,7 @@ import
org.apache.spark.sql.execution.datasources.v2.BatchScanExec
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.Utils
import org.apache.spark.util.collection.Utils.createArray
@@ -1266,6 +1267,7 @@ class OrcV1QuerySuite extends OrcQuerySuite {
.set(SQLConf.USE_V1_SOURCE_LIST, "orc")
}
+@ExtendedSQLTest
class OrcV2QuerySuite extends OrcQuerySuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
index 155fd592cf8a..58b98fe053ae 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
@@ -41,6 +41,7 @@ import
org.apache.spark.sql.execution.datasources.orc.OrcCompressionCodec._
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.Utils
case class OrcData(intField: Int, stringField: String)
@@ -1103,6 +1104,7 @@ class OrcSourceV1Suite extends OrcSourceSuite {
.set(SQLConf.USE_V1_SOURCE_LIST, "orc")
}
+@ExtendedSQLTest
class OrcSourceV2Suite extends OrcSourceSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormatSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormatSuite.scala
index b0ad5ca8cab7..01c8507032ce 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormatSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormatSuite.scala
@@ -30,6 +30,7 @@ import
org.apache.spark.sql.execution.datasources.CommonFileDataSourceSuite
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.HadoopFSUtils
abstract class ParquetFileFormatSuite
@@ -269,6 +270,7 @@ class ParquetFileFormatV1Suite extends
ParquetFileFormatSuite {
.set(SQLConf.USE_V1_SOURCE_LIST, "parquet")
}
+@ExtendedSQLTest
class ParquetFileFormatV2Suite extends ParquetFileFormatSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetGeoSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetGeoSuite.scala
index 057f722f215c..185ea357fd50 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetGeoSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetGeoSuite.scala
@@ -25,7 +25,9 @@ import org.apache.spark.sql.{DataFrame, Row}
import org.apache.spark.sql.functions.{st_asbinary, st_geogfromwkb,
st_geomfromwkb, st_srid}
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types.{Geography, GeographyType, StructField,
StructType}
+import org.apache.spark.tags.ExtendedSQLTest
+@ExtendedSQLTest
class ParquetGeoSuite
extends ParquetCompatibilityTest
with SharedSparkSession {
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala
index e36bb50416f3..bdb78b9e105d 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala
@@ -39,6 +39,7 @@ import
org.apache.spark.sql.execution.datasources.v2.parquet.ParquetScan
import org.apache.spark.sql.functions.struct
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.Utils
/**
@@ -1371,6 +1372,7 @@ class ParquetV1QuerySuite extends ParquetQuerySuite {
}
}
+@ExtendedSQLTest
class ParquetV2QuerySuite extends ParquetQuerySuite {
import testImplicits._
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetTypeWideningSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetTypeWideningSuite.scala
index 7ccd664f6c7c..ac33bd7ba336 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetTypeWideningSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetTypeWideningSuite.scala
@@ -34,7 +34,9 @@ import
org.apache.spark.sql.internal.SQLConf.ParquetOutputTimestampType
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
import org.apache.spark.sql.types.DecimalType.{ByteDecimal, IntDecimal,
LongDecimal, ShortDecimal}
+import org.apache.spark.tags.ExtendedSQLTest
+@ExtendedSQLTest
class ParquetTypeWideningSuite
extends ParquetTest
with SharedSparkSession
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
index 6778f7109a6e..38fea013a246 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
@@ -33,6 +33,7 @@ import
org.apache.spark.sql.execution.datasources.CommonFileDataSourceSuite
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types.{StringType, StructType}
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.Utils
abstract class TextSuite extends SharedSparkSession with
CommonFileDataSourceSuite {
@@ -349,6 +350,7 @@ class TextV1Suite extends TextSuite {
.set(SQLConf.USE_V1_SOURCE_LIST, "text")
}
+@ExtendedSQLTest
class TextV2Suite extends TextSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/WholeTextFileSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/WholeTextFileSuite.scala
index 81eddc182539..08a4a72ec67a 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/WholeTextFileSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/WholeTextFileSuite.scala
@@ -25,6 +25,7 @@ import
org.apache.spark.sql.catalyst.util.HadoopCompressionCodec.GZIP
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types.{StringType, StructType}
+import org.apache.spark.tags.ExtendedSQLTest
abstract class WholeTextFileSuite extends SharedSparkSession {
@@ -113,6 +114,7 @@ class WholeTextFileV1Suite extends WholeTextFileSuite {
.set(SQLConf.USE_V1_SOURCE_LIST, "text")
}
+@ExtendedSQLTest
class WholeTextFileV2Suite extends WholeTextFileSuite {
override protected def sparkConf: SparkConf =
super
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLLastAttemptMetricPlanShapesSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLLastAttemptMetricPlanShapesSuite.scala
index 033a0eea5b5a..7801188d532c 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLLastAttemptMetricPlanShapesSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLLastAttemptMetricPlanShapesSuite.scala
@@ -29,7 +29,9 @@ import org.apache.spark.sql.execution.exchange._
import org.apache.spark.sql.functions.udf
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.test.SharedSparkSession
+import org.apache.spark.tags.ExtendedSQLTest
+@ExtendedSQLTest
class SQLLastAttemptMetricPlanShapesSuite
extends SharedSparkSession
with SQLMetricsTestUtils
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StatePartitionAllColumnFamiliesWriterSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StatePartitionAllColumnFamiliesWriterSuite.scala
index 66617a4d0908..a6d9ad985efd 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StatePartitionAllColumnFamiliesWriterSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StatePartitionAllColumnFamiliesWriterSuite.scala
@@ -32,12 +32,14 @@ import org.apache.spark.sql.streaming.{InputEvent,
ListStateTTLProcessor, MapInp
import org.apache.spark.sql.streaming.util.{StreamManualClock,
TTLProcessorUtils}
import org.apache.spark.sql.streaming.util.{EventTimeTimerProcessor,
MultiStateVarProcessor, MultiStateVarProcessorTestUtils, TimerTestUtils}
import org.apache.spark.sql.types.StructType
+import org.apache.spark.tags.SlowSQLTest
/**
* Test suite for StatePartitionAllColumnFamiliesWriter.
* Tests the writer's ability to correctly write raw bytes read from
* StatePartitionAllColumnFamiliesReader to a state store without loading
previous versions.
*/
+@SlowSQLTest
class StatePartitionAllColumnFamiliesWriterSuite extends
StateDataSourceTestBase {
import testImplicits._
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreDecoupledMaintenanceSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreDecoupledMaintenanceSuite.scala
index a08b606b578b..cea84e80efd0 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreDecoupledMaintenanceSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreDecoupledMaintenanceSuite.scala
@@ -1244,6 +1244,7 @@ class StateStoreDecoupledMaintenanceSuiteWithRowChecksum
extends StateStoreDecoupledMaintenanceSuite
with EnableStateStoreRowChecksum
+@ExtendedSQLTest
class RocksDBDecoupledMaintenanceSuite
extends StateStoreDecoupledMaintenanceSuiteBase[RocksDBStateStoreProvider]
with AlsoTestWithEncodingTypes
@@ -1252,6 +1253,7 @@ class RocksDBDecoupledMaintenanceSuite
override def afterEach(): Unit = {}
}
+@ExtendedSQLTest
class RocksDBDecoupledMaintenanceSuiteWithRowChecksum
extends RocksDBDecoupledMaintenanceSuite
with EnableStateStoreRowChecksum
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala
index accf6439234a..4668bde453de 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala
@@ -34,6 +34,7 @@ import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.internal.SQLConf.PartitionOverwriteMode
import org.apache.spark.sql.test.SharedSparkSession
import org.apache.spark.sql.types._
+import org.apache.spark.tags.ExtendedSQLTest
import org.apache.spark.util.Utils
class SimpleInsertSource extends SchemaRelationProvider {
@@ -57,6 +58,7 @@ case class SimpleInsert(userSpecifiedSchema:
StructType)(@transient val sparkSes
}
}
+@ExtendedSQLTest
class InsertSuite extends DataSourceTest with SharedSparkSession {
import testImplicits._
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/streaming/RealTimeTransformWithStateSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/streaming/RealTimeTransformWithStateSuite.scala
index d7482d3e0225..eac33ef717e5 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/streaming/RealTimeTransformWithStateSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/streaming/RealTimeTransformWithStateSuite.scala
@@ -36,6 +36,7 @@ import org.apache.spark.sql.execution.streaming.state.{
import org.apache.spark.sql.functions.col
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.streaming.util.{GlobalSingletonManualClock,
StreamManualClock}
+import org.apache.spark.tags.SlowSQLTest
private class RealTimeEagerCountProcessor
extends StatefulProcessor[String, (String, Int), (String, Long)] {
@@ -571,6 +572,7 @@ private object RealTimeInitialStateBootstrapBlock {
}
}
+@SlowSQLTest
class RealTimeTransformWithStateSuite extends StreamRealTimeModeE2ESuiteBase {
import testImplicits._
@@ -2045,5 +2047,6 @@ class RealTimeTransformWithStateSuite extends
StreamRealTimeModeE2ESuiteBase {
}
}
+@SlowSQLTest
class RealTimeTransformWithStateSuiteWithRowChecksum
extends RealTimeTransformWithStateSuite with EnableStateStoreRowChecksum
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateInitialStateSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateInitialStateSuite.scala
index 9685f70b86a7..53a02b3f6d20 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateInitialStateSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateInitialStateSuite.scala
@@ -756,6 +756,7 @@ class TransformWithStateInitialStateSuite extends
StateStoreMetricsTest
}
}
+@SlowSQLTest
class TransformWithStateInitialStateSuiteCheckpointV2
extends TransformWithStateInitialStateSuite {
override def beforeAll(): Unit = {
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/StreamingSourceEvolutionSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/StreamingSourceEvolutionSuite.scala
index cdf8cb76d8e3..ee52768ceb5f 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/StreamingSourceEvolutionSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/streaming/test/StreamingSourceEvolutionSuite.scala
@@ -29,12 +29,14 @@ import
org.apache.spark.sql.execution.streaming.checkpointing.{OffsetMap, Offset
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.streaming.StreamTest
import org.apache.spark.sql.streaming.Trigger._
+import org.apache.spark.tags.SlowSQLTest
import org.apache.spark.util.Utils
/**
* Test suite for streaming source naming and validation.
* Tests cover the naming API, validation rules, and resolution pipeline.
*/
+@SlowSQLTest
class StreamingSourceEvolutionSuite extends StreamTest {
private def newMetadataDir =
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
