This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4c800478d5a [SPARK-43830][BUILD] Update scalatest and scalatestplus
related dependencies to newest version
4c800478d5a is described below
commit 4c800478d5a6c76dca3ed1fc945de71182fd65e3
Author: panbingkun <[email protected]>
AuthorDate: Sat May 27 18:12:52 2023 -0500
[SPARK-43830][BUILD] Update scalatest and scalatestplus related
dependencies to newest version
### What changes were proposed in this pull request?
The pr aims to update scalatest and scalatestplus related dependencies to
newest version, include:
This pr aims upgrade `scalatest` related test dependencies to 3.2.16:
- scalatest: upgrade scalatest from 3.2.15 to 3.2.16
- mockito
- mockito-core: upgrade from 4.6.1 to 4.11.0
- mockito-inline: upgrade from 4.6.1 to 4.11.0
- selenium-java: upgrade from 4.7.2 to 4.9.1
- htmlunit-driver: upgrade from 4.7.2 to 4.9.1
- htmlunit: upgrade from 2.67.0 to 2.70.0
- scalatestplus
- scalacheck-1-17: upgrade from 3.2.15.0 to 3.2.16.0
- mockito: upgrade from `mockito-4-6` 3.2.15.0 to `mockito-4-11` 3.2.16.0
- selenium: upgrade from `selenium-4-7` 3.2.15.0 to `selenium-4-9`
3.2.16.0
### Why are the changes needed?
The relevant release notes as follows:
- scalatest:
-
https://github.com/scalatest/scalatest/releases/tag/release-3.2.16
- [mockito](https://github.com/mockito/mockito)
- https://github.com/mockito/mockito/releases/tag/v4.11.0
- https://github.com/mockito/mockito/releases/tag/v4.10.0
- https://github.com/mockito/mockito/releases/tag/v4.9.0
- https://github.com/mockito/mockito/releases/tag/v4.8.1
- https://github.com/mockito/mockito/releases/tag/v4.8.0
- https://github.com/mockito/mockito/releases/tag/v4.7.0
- [selenium-java](https://github.com/SeleniumHQ/selenium)
- https://github.com/SeleniumHQ/selenium/releases/tag/selenium-4.9.1
- https://github.com/SeleniumHQ/selenium/releases/tag/selenium-4.9.0
- https://github.com/SeleniumHQ/selenium/releases/tag/selenium-4.8.3-java
- https://github.com/SeleniumHQ/selenium/releases/tag/selenium-4.8.2-java
- https://github.com/SeleniumHQ/selenium/releases/tag/selenium-4.8.1
- https://github.com/SeleniumHQ/selenium/releases/tag/selenium-4.8.0
- [htmlunit-driver](https://github.com/SeleniumHQ/htmlunit-driver)
-
https://github.com/SeleniumHQ/htmlunit-driver/releases/tag/htmlunit-driver-4.9.1
-
https://github.com/SeleniumHQ/htmlunit-driver/releases/tag/htmlunit-driver-4.9.0
-
https://github.com/SeleniumHQ/htmlunit-driver/releases/tag/htmlunit-driver-4.8.3
-
https://github.com/SeleniumHQ/htmlunit-driver/releases/tag/htmlunit-driver-4.8.1.1
- https://github.com/SeleniumHQ/htmlunit-driver/releases/tag/4.8.1
- https://github.com/SeleniumHQ/htmlunit-driver/releases/tag/4.8.0
- [htmlunit](https://github.com/HtmlUnit/htmlunit)
- https://github.com/HtmlUnit/htmlunit/releases/tag/2.70.0
- Why this version: because the 4.9.1 version of Selenium relies on it.
https://github.com/SeleniumHQ/selenium/blob/selenium-4.9.1/java/maven_deps.bzl#L83
-
[org.scalatestplus:scalacheck-1-17](https://github.com/scalatest/scalatestplus-scalacheck)
-
https://github.com/scalatest/scalatestplus-scalacheck/releases/tag/release-3.2.16.0-for-scalacheck-1.17
-
[org.scalatestplus:mockito-4-11](https://github.com/scalatest/scalatestplus-mockito)
-
https://github.com/scalatest/scalatestplus-mockito/releases/tag/release-3.2.16.0-for-mockito-4.11
-
[org.scalatestplus:selenium-4-9](https://github.com/scalatest/scalatestplus-selenium)
-
https://github.com/scalatest/scalatestplus-selenium/releases/tag/release-3.2.16.0-for-selenium-4.9
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- Pass GitHub Actions
- Manual test:
- ChromeUISeleniumSuite
- RocksDBBackendChromeUIHistoryServerSuite
```
build/sbt -Dguava.version=31.1-jre
-Dspark.test.webdriver.chrome.driver=/path/to/chromedriver
-Dtest.default.exclude.tags="" -Phive -Phive-thriftserver "core/testOnly
org.apache.spark.ui.ChromeUISeleniumSuite"
build/sbt -Dguava.version=31.1-jre
-Dspark.test.webdriver.chrome.driver=/path/to/chromedriver
-Dtest.default.exclude.tags="" -Phive -Phive-thriftserver "core/testOnly
org.apache.spark.deploy.history.RocksDBBackendChromeUIHistoryServerSuite"
```
<img width="856" alt="image"
src="https://github.com/apache/spark/assets/15246973/73349ffb-4198-4371-a741-411712d14712";>
Closes #41341 from panbingkun/upgrade_scalatest.
Authored-by: panbingkun <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
---
.../org/apache/spark/HeartbeatReceiverSuite.scala | 2 +-
.../org/apache/spark/api/java/JavaUtilsSuite.scala | 4 +++-
.../deploy/history/FsHistoryProviderSuite.scala | 10 ++++-----
.../history/HistoryServerDiskManagerSuite.scala | 3 ++-
.../spark/deploy/worker/DriverRunnerTest.scala | 2 +-
.../CoarseGrainedExecutorBackendSuite.scala | 6 +++--
.../spark/executor/ProcfsMetricsGetterSuite.scala | 2 +-
.../internal/plugin/PluginContainerSuite.scala | 4 ++--
.../apache/spark/scheduler/DAGSchedulerSuite.scala | 7 +++---
.../scheduler/OutputCommitCoordinatorSuite.scala | 6 +++--
.../spark/scheduler/TaskResultGetterSuite.scala | 4 ++--
.../spark/scheduler/TaskSchedulerImplSuite.scala | 4 ++--
.../spark/scheduler/TaskSetManagerSuite.scala | 6 ++---
.../storage/BlockManagerReplicationSuite.scala | 2 +-
.../apache/spark/storage/BlockManagerSuite.scala | 12 +++++-----
.../storage/PartiallySerializedBlockSuite.scala | 14 +++++++-----
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 26 +++++++++++-----------
.../KubernetesClusterSchedulerBackendSuite.scala | 2 +-
.../org/apache/spark/deploy/yarn/ClientSuite.scala | 7 +++---
.../spark/deploy/yarn/YarnAllocatorSuite.scala | 4 ++--
.../analysis/AnalysisExternalCatalogSuite.scala | 4 ++--
.../catalyst/analysis/TableLookupCacheSuite.scala | 4 ++--
.../scala/org/apache/spark/sql/JoinSuite.scala | 2 +-
.../sql/errors/QueryExecutionErrorsSuite.scala | 6 ++---
.../SparkExecuteStatementOperationSuite.scala | 2 +-
.../streaming/ReceivedBlockTrackerSuite.scala | 2 +-
27 files changed, 81 insertions(+), 68 deletions(-)
diff --git a/core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
b/core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
index 879ce558406..ee0a5773692 100644
--- a/core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
+++ b/core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
@@ -73,7 +73,7 @@ class HeartbeatReceiverSuite
.setMaster("local[2]")
.setAppName("test")
.set(DYN_ALLOCATION_TESTING, true)
- sc = spy(new SparkContext(conf))
+ sc = spy[SparkContext](new SparkContext(conf))
scheduler = mock(classOf[TaskSchedulerImpl])
when(sc.taskScheduler).thenReturn(scheduler)
when(scheduler.excludedNodes).thenReturn(Predef.Set[String]())
diff --git a/core/src/test/scala/org/apache/spark/api/java/JavaUtilsSuite.scala
b/core/src/test/scala/org/apache/spark/api/java/JavaUtilsSuite.scala
index 8e6e3e09686..ee20d51b892 100644
--- a/core/src/test/scala/org/apache/spark/api/java/JavaUtilsSuite.scala
+++ b/core/src/test/scala/org/apache/spark/api/java/JavaUtilsSuite.scala
@@ -22,6 +22,7 @@ import java.io.Serializable
import org.mockito.Mockito._
import org.apache.spark.SparkFunSuite
+import org.apache.spark.api.java.JavaUtils.SerializableMapWrapper
class JavaUtilsSuite extends SparkFunSuite {
@@ -33,7 +34,8 @@ class JavaUtilsSuite extends SparkFunSuite {
src.put(key, "42")
- val map: java.util.Map[Double, String] =
spy(JavaUtils.mapAsSerializableJavaMap(src))
+ val map: java.util.Map[Double, String] =
spy[SerializableMapWrapper[Double, String]](
+ JavaUtils.mapAsSerializableJavaMap(src))
assert(map.containsKey(key))
diff --git
a/core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
b/core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
index 4e026486e84..893f1083357 100644
---
a/core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
+++
b/core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
@@ -788,7 +788,7 @@ abstract class FsHistoryProviderSuite extends SparkFunSuite
with Matchers with P
}
test("provider correctly checks whether fs is in safe mode") {
- val provider = spy(new FsHistoryProvider(createTestConf()))
+ val provider = spy[FsHistoryProvider](new
FsHistoryProvider(createTestConf()))
val dfs = mock(classOf[DistributedFileSystem])
// Asserts that safe mode is false because we can't really control the
return value of the mock,
// since the API is different between hadoop 1 and 2.
@@ -1032,7 +1032,7 @@ abstract class FsHistoryProviderSuite extends
SparkFunSuite with Matchers with P
withTempDir { storeDir =>
val conf = createTestConf().set(LOCAL_STORE_DIR,
storeDir.getAbsolutePath())
val clock = new ManualClock()
- val provider = spy(new FsHistoryProvider(conf, clock))
+ val provider = spy[FsHistoryProvider](new FsHistoryProvider(conf, clock))
val appId = "new1"
// Write logs for two app attempts.
@@ -1196,11 +1196,11 @@ abstract class FsHistoryProviderSuite extends
SparkFunSuite with Matchers with P
SparkListenerApplicationStart("accessGranted", Some("accessGranted"),
1L, "test", None),
SparkListenerApplicationEnd(5L))
var isReadable = false
- val mockedFs = spy(provider.fs)
+ val mockedFs = spy[FileSystem](provider.fs)
doThrow(new AccessControlException("Cannot read accessDenied
file")).when(mockedFs).open(
argThat((path: Path) => path.getName.toLowerCase(Locale.ROOT) ==
"accessdenied" &&
!isReadable))
- val mockedProvider = spy(provider)
+ val mockedProvider = spy[FsHistoryProvider](provider)
when(mockedProvider.fs).thenReturn(mockedFs)
updateAndCheck(mockedProvider) { list =>
list.size should be(1)
@@ -1225,7 +1225,7 @@ abstract class FsHistoryProviderSuite extends
SparkFunSuite with Matchers with P
test("check in-progress event logs absolute length") {
val path = new Path("testapp.inprogress")
val provider = new FsHistoryProvider(createTestConf())
- val mockedProvider = spy(provider)
+ val mockedProvider = spy[FsHistoryProvider](provider)
val mockedFs = mock(classOf[FileSystem])
val in = mock(classOf[FSDataInputStream])
val dfsIn = mock(classOf[DFSInputStream])
diff --git
a/core/src/test/scala/org/apache/spark/deploy/history/HistoryServerDiskManagerSuite.scala
b/core/src/test/scala/org/apache/spark/deploy/history/HistoryServerDiskManagerSuite.scala
index 373d1c557fc..e4248a49b90 100644
---
a/core/src/test/scala/org/apache/spark/deploy/history/HistoryServerDiskManagerSuite.scala
+++
b/core/src/test/scala/org/apache/spark/deploy/history/HistoryServerDiskManagerSuite.scala
@@ -62,7 +62,8 @@ abstract class HistoryServerDiskManagerSuite extends
SparkFunSuite with BeforeAn
private def mockManager(): HistoryServerDiskManager = {
val conf = new SparkConf().set(MAX_LOCAL_DISK_USAGE, MAX_USAGE)
- val manager = spy(new HistoryServerDiskManager(conf, testDir, store, new
ManualClock()))
+ val manager = spy[HistoryServerDiskManager](
+ new HistoryServerDiskManager(conf, testDir, store, new ManualClock()))
doAnswer(AdditionalAnswers.returnsFirstArg[Long]()).when(manager)
.approximateSize(anyLong(), anyBoolean())
manager
diff --git
a/core/src/test/scala/org/apache/spark/deploy/worker/DriverRunnerTest.scala
b/core/src/test/scala/org/apache/spark/deploy/worker/DriverRunnerTest.scala
index e429ddfd570..e97196084e0 100644
--- a/core/src/test/scala/org/apache/spark/deploy/worker/DriverRunnerTest.scala
+++ b/core/src/test/scala/org/apache/spark/deploy/worker/DriverRunnerTest.scala
@@ -39,7 +39,7 @@ class DriverRunnerTest extends SparkFunSuite {
val conf = new SparkConf()
val worker = mock(classOf[RpcEndpointRef])
doNothing().when(worker).send(any())
- spy(new DriverRunner(conf, "driverId", new File("workDir"), new
File("sparkHome"),
+ spy[DriverRunner](new DriverRunner(conf, "driverId", new File("workDir"),
new File("sparkHome"),
driverDescription, worker, "spark://1.2.3.4/worker/",
"http://publicAddress:80";,
new SecurityManager(conf)))
}
diff --git
a/core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala
b/core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala
index 7b8b7cf4cdd..7ba5dd4793b 100644
---
a/core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala
+++
b/core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala
@@ -408,7 +408,8 @@ class CoarseGrainedExecutorBackendSuite extends
SparkFunSuite
val executor = backend.executor
// Mock the executor.
when(executor.threadPool).thenReturn(threadPool)
- val runningTasks = spy(new ConcurrentHashMap[Long, Executor#TaskRunner])
+ val runningTasks = spy[ConcurrentHashMap[Long, Executor#TaskRunner]](
+ new ConcurrentHashMap[Long, Executor#TaskRunner])
when(executor.runningTasks).thenAnswer(_ => runningTasks)
when(executor.conf).thenReturn(conf)
@@ -496,7 +497,8 @@ class CoarseGrainedExecutorBackendSuite extends
SparkFunSuite
val executor = backend.executor
// Mock the executor.
when(executor.threadPool).thenReturn(threadPool)
- val runningTasks = spy(new ConcurrentHashMap[Long, Executor#TaskRunner])
+ val runningTasks = spy[ConcurrentHashMap[Long, Executor#TaskRunner]](
+ new ConcurrentHashMap[Long, Executor#TaskRunner])
when(executor.runningTasks).thenAnswer(_ => runningTasks)
when(executor.conf).thenReturn(conf)
diff --git
a/core/src/test/scala/org/apache/spark/executor/ProcfsMetricsGetterSuite.scala
b/core/src/test/scala/org/apache/spark/executor/ProcfsMetricsGetterSuite.scala
index ff0374da1bc..d583afdf07c 100644
---
a/core/src/test/scala/org/apache/spark/executor/ProcfsMetricsGetterSuite.scala
+++
b/core/src/test/scala/org/apache/spark/executor/ProcfsMetricsGetterSuite.scala
@@ -43,7 +43,7 @@ class ProcfsMetricsGetterSuite extends SparkFunSuite {
test("SPARK-34845: partial metrics shouldn't be returned") {
val p = new ProcfsMetricsGetter(getTestResourcePath("ProcfsMetrics"))
- val mockedP = spy(p)
+ val mockedP = spy[ProcfsMetricsGetter](p)
var ptree: Set[Int] = Set(26109, 22763)
when(mockedP.computeProcessTree).thenReturn(ptree)
diff --git
a/core/src/test/scala/org/apache/spark/internal/plugin/PluginContainerSuite.scala
b/core/src/test/scala/org/apache/spark/internal/plugin/PluginContainerSuite.scala
index 2b8515d52d1..e7959c8f742 100644
---
a/core/src/test/scala/org/apache/spark/internal/plugin/PluginContainerSuite.scala
+++
b/core/src/test/scala/org/apache/spark/internal/plugin/PluginContainerSuite.scala
@@ -307,14 +307,14 @@ class TestSparkPlugin extends SparkPlugin {
override def driverPlugin(): DriverPlugin = {
val p = new TestDriverPlugin()
require(TestSparkPlugin.driverPlugin == null, "Driver plugin already
initialized.")
- TestSparkPlugin.driverPlugin = spy(p)
+ TestSparkPlugin.driverPlugin = spy[TestDriverPlugin](p)
TestSparkPlugin.driverPlugin
}
override def executorPlugin(): ExecutorPlugin = {
val p = new TestExecutorPlugin()
require(TestSparkPlugin.executorPlugin == null, "Executor plugin already
initialized.")
- TestSparkPlugin.executorPlugin = spy(p)
+ TestSparkPlugin.executorPlugin = spy[TestExecutorPlugin](p)
TestSparkPlugin.executorPlugin
}
diff --git
a/core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
b/core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
index 34bc8e31bbd..73ee879ad53 100644
--- a/core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
+++ b/core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
@@ -402,10 +402,11 @@ class DAGSchedulerSuite extends SparkFunSuite with
TempLocalSparkContext with Ti
results.clear()
securityMgr = new SecurityManager(sc.getConf)
broadcastManager = new BroadcastManager(true, sc.getConf)
- mapOutputTracker = spy(new MyMapOutputTrackerMaster(sc.getConf,
broadcastManager))
- blockManagerMaster = spy(new MyBlockManagerMaster(sc.getConf))
+ mapOutputTracker = spy[MyMapOutputTrackerMaster](
+ new MyMapOutputTrackerMaster(sc.getConf, broadcastManager))
+ blockManagerMaster = spy[MyBlockManagerMaster](new
MyBlockManagerMaster(sc.getConf))
doNothing().when(blockManagerMaster).updateRDDBlockVisibility(any(), any())
- scheduler = spy(new MyDAGScheduler(
+ scheduler = spy[MyDAGScheduler](new MyDAGScheduler(
sc,
taskScheduler,
sc.listenerBus,
diff --git
a/core/src/test/scala/org/apache/spark/scheduler/OutputCommitCoordinatorSuite.scala
b/core/src/test/scala/org/apache/spark/scheduler/OutputCommitCoordinatorSuite.scala
index 95e2429ea58..44dc9a5f97d 100644
---
a/core/src/test/scala/org/apache/spark/scheduler/OutputCommitCoordinatorSuite.scala
+++
b/core/src/test/scala/org/apache/spark/scheduler/OutputCommitCoordinatorSuite.scala
@@ -87,7 +87,8 @@ class OutputCommitCoordinatorSuite extends SparkFunSuite with
BeforeAndAfter {
isLocal: Boolean,
listenerBus: LiveListenerBus): SparkEnv = {
outputCommitCoordinator =
- spy(new OutputCommitCoordinator(conf, isDriver = true, Option(this)))
+ spy[OutputCommitCoordinator](
+ new OutputCommitCoordinator(conf, isDriver = true, Option(this)))
// Use Mockito.spy() to maintain the default infrastructure everywhere
else.
// This mocking allows us to control the coordinator responses in test
cases.
SparkEnv.createDriverEnv(conf, isLocal, listenerBus,
@@ -95,7 +96,8 @@ class OutputCommitCoordinatorSuite extends SparkFunSuite with
BeforeAndAfter {
}
}
// Use Mockito.spy() to maintain the default infrastructure everywhere else
- val mockTaskScheduler =
spy(sc.taskScheduler.asInstanceOf[TaskSchedulerImpl])
+ val mockTaskScheduler = spy[TaskSchedulerImpl](
+ sc.taskScheduler.asInstanceOf[TaskSchedulerImpl])
doAnswer { (invoke: InvocationOnMock) =>
// Submit the tasks, then force the task scheduler to dequeue the
diff --git
a/core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala
b/core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala
index 1f61fab3e07..3ea084307cb 100644
--- a/core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala
+++ b/core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala
@@ -145,7 +145,7 @@ class TaskResultGetterSuite extends SparkFunSuite with
BeforeAndAfter with Local
test("handling total size of results larger than maxResultSize") {
sc = new SparkContext("local", "test", conf)
val scheduler = new DummyTaskSchedulerImpl(sc)
- val spyScheduler = spy(scheduler)
+ val spyScheduler = spy[DummyTaskSchedulerImpl](scheduler)
val resultGetter = new TaskResultGetter(sc.env, spyScheduler)
scheduler.taskResultGetter = resultGetter
val myTsm = new TaskSetManager(spyScheduler, FakeTask.createTaskSet(2), 1)
{
@@ -258,7 +258,7 @@ class TaskResultGetterSuite extends SparkFunSuite with
BeforeAndAfter with Local
// Set up custom TaskResultGetter and TaskSchedulerImpl spy
sc = new SparkContext("local", "test", conf)
val scheduler = sc.taskScheduler.asInstanceOf[TaskSchedulerImpl]
- val spyScheduler = spy(scheduler)
+ val spyScheduler = spy[TaskSchedulerImpl](scheduler)
val resultGetter = new MyTaskResultGetter(sc.env, spyScheduler)
val newDAGScheduler = new DAGScheduler(sc, spyScheduler)
scheduler.taskResultGetter = resultGetter
diff --git
a/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala
b/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala
index a484dae6f80..7d2b4f5221a 100644
---
a/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala
+++
b/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala
@@ -109,7 +109,7 @@ class TaskSchedulerImplSuite extends SparkFunSuite with
LocalSparkContext
override def createTaskSetManager(taskSet: TaskSet, maxFailures: Int):
TaskSetManager = {
val tsm = super.createTaskSetManager(taskSet, maxFailures)
// we need to create a spied tsm just so we can set the
TaskSetExcludelist
- val tsmSpy = spy(tsm)
+ val tsmSpy = spy[TaskSetManager](tsm)
val taskSetExcludelist = mock[TaskSetExcludelist]
when(tsmSpy.taskSetExcludelistHelperOpt).thenReturn(Some(taskSetExcludelist))
stageToMockTaskSetManager(taskSet.stageId) = tsmSpy
@@ -1946,7 +1946,7 @@ class TaskSchedulerImplSuite extends SparkFunSuite with
LocalSparkContext
override def createTaskSetManager(taskSet: TaskSet, maxFailures: Int):
TaskSetManager = {
val tsm = super.createTaskSetManager(taskSet, maxFailures)
// we need to create a spied tsm so that we can see the copies running
- val tsmSpy = spy(tsm)
+ val tsmSpy = spy[TaskSetManager](tsm)
stageToMockTaskSetManager(taskSet.stageId) = tsmSpy
tsmSpy
}
diff --git
a/core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
b/core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
index 45360f486ed..cb70dbb0289 100644
--- a/core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
+++ b/core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
@@ -389,7 +389,7 @@ class TaskSetManagerSuite
manager.isZombie = false
// offers not accepted due to excludelist are not delay schedule rejects
- val tsmSpy = spy(manager)
+ val tsmSpy = spy[TaskSetManager](manager)
val excludelist = mock(classOf[TaskSetExcludelist])
when(tsmSpy.taskSetExcludelistHelperOpt).thenReturn(Some(excludelist))
when(excludelist.isNodeExcludedForTaskSet(any())).thenReturn(true)
@@ -1416,7 +1416,7 @@ class TaskSetManagerSuite
val taskSet = FakeTask.createTaskSet(4)
val tsm = new TaskSetManager(sched, taskSet, 4)
// we need a spy so we can attach our mock excludelist
- val tsmSpy = spy(tsm)
+ val tsmSpy = spy[TaskSetManager](tsm)
val excludelist = mock(classOf[TaskSetExcludelist])
when(tsmSpy.taskSetExcludelistHelperOpt).thenReturn(Some(excludelist))
@@ -1497,7 +1497,7 @@ class TaskSetManagerSuite
val mockListenerBus = mock(classOf[LiveListenerBus])
val healthTracker = new HealthTracker(mockListenerBus, conf, None, clock)
val taskSetManager = new TaskSetManager(sched, taskSet, 1,
Some(healthTracker))
- val taskSetManagerSpy = spy(taskSetManager)
+ val taskSetManagerSpy = spy[TaskSetManager](taskSetManager)
val taskDesc = taskSetManagerSpy.resourceOffer(exec, host,
TaskLocality.ANY)._1
diff --git
a/core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala
b/core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala
index 8729ae1edfb..38a669bc857 100644
---
a/core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala
+++
b/core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala
@@ -275,7 +275,7 @@ trait BlockManagerReplicationBehavior extends SparkFunSuite
// create 1 faulty block manager by injecting faulty memory manager
val memManager = UnifiedMemoryManager(conf, numCores = 1)
- val mockedMemoryManager = spy(memManager)
+ val mockedMemoryManager = spy[UnifiedMemoryManager](memManager)
doAnswer(_ =>
false).when(mockedMemoryManager).acquireStorageMemory(any(), any(), any())
val store2 = makeBlockManager(10000, "host-2", Some(mockedMemoryManager))
diff --git
a/core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala
b/core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala
index 29592434765..ab6c2693b0e 100644
--- a/core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala
+++ b/core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala
@@ -186,8 +186,8 @@ class BlockManagerSuite extends SparkFunSuite with Matchers
with PrivateMethodTe
when(sc.conf).thenReturn(conf)
val blockManagerInfo = new mutable.HashMap[BlockManagerId,
BlockManagerInfo]()
- liveListenerBus = spy(new LiveListenerBus(conf))
- master = spy(new BlockManagerMaster(rpcEnv.setupEndpoint("blockmanager",
+ liveListenerBus = spy[LiveListenerBus](new LiveListenerBus(conf))
+ master = spy[BlockManagerMaster](new
BlockManagerMaster(rpcEnv.setupEndpoint("blockmanager",
new BlockManagerMasterEndpoint(rpcEnv, true, conf,
liveListenerBus, None, blockManagerInfo, mapOutputTracker,
shuffleManager,
isDriver = true)),
@@ -873,7 +873,7 @@ class BlockManagerSuite extends SparkFunSuite with Matchers
with PrivateMethodTe
conf.set("spark.shuffle.io.maxRetries", "0")
val sameHostBm = makeBlockManager(8000, "sameHost", master)
- val otherHostTransferSrv = spy(sameHostBm.blockTransferService)
+ val otherHostTransferSrv =
spy[BlockTransferService](sameHostBm.blockTransferService)
doAnswer { _ =>
"otherHost"
}.when(otherHostTransferSrv).hostName
@@ -888,7 +888,7 @@ class BlockManagerSuite extends SparkFunSuite with Matchers
with PrivateMethodTe
val blockId = "list"
bmToPutBlock.putIterator(blockId, List(array).iterator, storageLevel,
tellMaster = true)
- val sameHostTransferSrv = spy(sameHostBm.blockTransferService)
+ val sameHostTransferSrv =
spy[BlockTransferService](sameHostBm.blockTransferService)
doAnswer { _ =>
fail("Fetching over network is not expected when the block is
requested from same host")
}.when(sameHostTransferSrv).fetchBlockSync(mc.any(), mc.any(), mc.any(),
mc.any(), mc.any())
@@ -935,7 +935,7 @@ class BlockManagerSuite extends SparkFunSuite with Matchers
with PrivateMethodTe
}
}
val store1 = makeBlockManager(8000, "executor1", this.master,
Some(mockTransferService))
- val spiedStore1 = spy(store1)
+ val spiedStore1 = spy[BlockManager](store1)
doAnswer { inv =>
val blockId = inv.getArguments()(0).asInstanceOf[BlockId]
val localDirs = inv.getArguments()(1).asInstanceOf[Array[String]]
@@ -974,7 +974,7 @@ class BlockManagerSuite extends SparkFunSuite with Matchers
with PrivateMethodTe
}
test("SPARK-14252: getOrElseUpdate should still read from remote storage") {
- val store = spy(makeBlockManager(8000, "executor1"))
+ val store = spy[BlockManager](makeBlockManager(8000, "executor1"))
val store2 = makeBlockManager(8000, "executor2")
val list1 = List(new Array[Byte](4000))
val blockId = RDDBlockId(0, 0)
diff --git
a/core/src/test/scala/org/apache/spark/storage/PartiallySerializedBlockSuite.scala
b/core/src/test/scala/org/apache/spark/storage/PartiallySerializedBlockSuite.scala
index 8177ef6e140..9753b483153 100644
---
a/core/src/test/scala/org/apache/spark/storage/PartiallySerializedBlockSuite.scala
+++
b/core/src/test/scala/org/apache/spark/storage/PartiallySerializedBlockSuite.scala
@@ -58,18 +58,22 @@ class PartiallySerializedBlockSuite
numItemsToBuffer: Int): PartiallySerializedBlock[T] = {
val bbos: ChunkedByteBufferOutputStream = {
- val spy = Mockito.spy(new ChunkedByteBufferOutputStream(128,
ByteBuffer.allocate))
+ val spy = Mockito.spy[ChunkedByteBufferOutputStream](
+ new ChunkedByteBufferOutputStream(128, ByteBuffer.allocate))
Mockito.doAnswer { (invocationOnMock: InvocationOnMock) =>
-
Mockito.spy(invocationOnMock.callRealMethod().asInstanceOf[ChunkedByteBuffer])
+ Mockito.spy[ChunkedByteBuffer](
+ invocationOnMock.callRealMethod().asInstanceOf[ChunkedByteBuffer])
}.when(spy).toChunkedByteBuffer
spy
}
val serializer = serializerManager
.getSerializer(implicitly[ClassTag[T]], autoPick = true).newInstance()
- val redirectableOutputStream = Mockito.spy(new RedirectableOutputStream)
+ val redirectableOutputStream = Mockito.spy[RedirectableOutputStream](
+ new RedirectableOutputStream)
redirectableOutputStream.setOutputStream(bbos)
- val serializationStream =
Mockito.spy(serializer.serializeStream(redirectableOutputStream))
+ val serializationStream = Mockito.spy[SerializationStream](
+ serializer.serializeStream(redirectableOutputStream))
(1 to numItemsToBuffer).foreach { _ =>
assert(iter.hasNext)
@@ -170,7 +174,7 @@ class PartiallySerializedBlockSuite
test(s"$testCaseName with finishWritingToStream() and numBuffered =
$numItemsToBuffer") {
val partiallySerializedBlock = partiallyUnroll(items.iterator,
numItemsToBuffer)
- val bbos = Mockito.spy(new ByteBufferOutputStream())
+ val bbos = Mockito.spy[ByteBufferOutputStream](new
ByteBufferOutputStream())
partiallySerializedBlock.finishWritingToStream(bbos)
Mockito.verify(memoryStore).releaseUnrollMemoryForThisTask(
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 9f6a8f2573b..d275c827ef1 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -201,7 +201,7 @@
netty-transport-native-kqueue/4.1.92.Final/osx-aarch_64/netty-transport-native-k
netty-transport-native-kqueue/4.1.92.Final/osx-x86_64/netty-transport-native-kqueue-4.1.92.Final-osx-x86_64.jar
netty-transport-native-unix-common/4.1.92.Final//netty-transport-native-unix-common-4.1.92.Final.jar
netty-transport/4.1.92.Final//netty-transport-4.1.92.Final.jar
-objenesis/3.2//objenesis-3.2.jar
+objenesis/3.3//objenesis-3.3.jar
okhttp/3.12.12//okhttp-3.12.12.jar
okio/1.15.0//okio-1.15.0.jar
opencsv/2.3//opencsv-2.3.jar
diff --git a/pom.xml b/pom.xml
index bc6a49c44c2..2394e429218 100644
--- a/pom.xml
+++ b/pom.xml
@@ -203,9 +203,9 @@
<!-- Please don't upgrade the version to 4.10+, it depends on JDK 11 -->
<antlr4.version>4.9.3</antlr4.version>
<jpam.version>1.1</jpam.version>
- <selenium.version>4.7.2</selenium.version>
- <htmlunit-driver.version>4.7.2</htmlunit-driver.version>
- <htmlunit.version>2.67.0</htmlunit.version>
+ <selenium.version>4.9.1</selenium.version>
+ <htmlunit-driver.version>4.9.1</htmlunit-driver.version>
+ <htmlunit.version>2.70.0</htmlunit.version>
<maven-antrun.version>3.1.0</maven-antrun.version>
<commons-crypto.version>1.1.0</commons-crypto.version>
<commons-cli.version>1.5.0</commons-cli.version>
@@ -390,12 +390,12 @@
</dependency>
<dependency>
<groupId>org.scalatestplus</groupId>
- <artifactId>mockito-4-6_${scala.binary.version}</artifactId>
+ <artifactId>mockito-4-11_${scala.binary.version}</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatestplus</groupId>
- <artifactId>selenium-4-7_${scala.binary.version}</artifactId>
+ <artifactId>selenium-4-9_${scala.binary.version}</artifactId>
<scope>test</scope>
</dependency>
<dependency>
@@ -1100,37 +1100,37 @@
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.binary.version}</artifactId>
- <version>3.2.15</version>
+ <version>3.2.16</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatestplus</groupId>
<artifactId>scalacheck-1-17_${scala.binary.version}</artifactId>
- <version>3.2.15.0</version>
+ <version>3.2.16.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatestplus</groupId>
- <artifactId>mockito-4-6_${scala.binary.version}</artifactId>
- <version>3.2.15.0</version>
+ <artifactId>mockito-4-11_${scala.binary.version}</artifactId>
+ <version>3.2.16.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatestplus</groupId>
- <artifactId>selenium-4-7_${scala.binary.version}</artifactId>
- <version>3.2.15.0</version>
+ <artifactId>selenium-4-9_${scala.binary.version}</artifactId>
+ <version>3.2.16.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
- <version>4.6.1</version>
+ <version>4.11.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
- <version>4.6.1</version>
+ <version>4.11.0</version>
<scope>test</scope>
</dependency>
<dependency>
diff --git
a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackendSuite.scala
b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackendSuite.scala
index bb5e93c92ac..b2e4a7182a7 100644
---
a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackendSuite.scala
+++
b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackendSuite.scala
@@ -174,7 +174,7 @@ class KubernetesClusterSchedulerBackendSuite extends
SparkFunSuite with BeforeAn
}
test("Remove executor") {
- val backend = spy(schedulerBackendUnderTest)
+ val backend =
spy[KubernetesClusterSchedulerBackend](schedulerBackendUnderTest)
when(backend.isExecutorActive(any())).thenReturn(false)
when(backend.isExecutorActive(mockitoEq("2"))).thenReturn(true)
diff --git
a/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala
b/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala
index 512359d0d6b..ba116c27716 100644
---
a/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala
+++
b/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala
@@ -228,7 +228,8 @@ class ClientSuite extends SparkFunSuite with Matchers {
val sparkConf = new SparkConf().set("spark.yarn.applicationType",
sourceType)
val args = new ClientArguments(Array())
- val appContext =
spy(Records.newRecord(classOf[ApplicationSubmissionContext]))
+ val appContext = spy[ApplicationSubmissionContext](
+ Records.newRecord(classOf[ApplicationSubmissionContext]))
val appId = ApplicationId.newInstance(123456, id)
appContext.setApplicationId(appId)
val getNewApplicationResponse =
Records.newRecord(classOf[GetNewApplicationResponse])
@@ -248,7 +249,7 @@ class ClientSuite extends SparkFunSuite with Matchers {
request.setApplicationSubmissionContext(subContext)
val rmContext = mock(classOf[RMContext])
- val conf = spy(classOf[Configuration])
+ val conf = spy[Configuration](classOf[Configuration])
val map = new ConcurrentHashMap[ApplicationId, RMApp]()
when(rmContext.getRMApps).thenReturn(map)
val dispatcher = mock(classOf[Dispatcher])
@@ -732,7 +733,7 @@ class ClientSuite extends SparkFunSuite with Matchers {
sparkConf: SparkConf,
args: Array[String] = Array()): Client = {
val clientArgs = new ClientArguments(args)
- spy(new Client(clientArgs, sparkConf, null))
+ spy[Client](new Client(clientArgs, sparkConf, null))
}
private def classpath(client: Client): Array[String] = {
diff --git
a/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
b/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
index 88c08abdca3..ed591fd9e36 100644
---
a/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
+++
b/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
@@ -638,7 +638,7 @@ class YarnAllocatorSuite extends SparkFunSuite with
Matchers {
}
test("SPARK-26269: YarnAllocator should have same excludeOnFailure behaviour
with YARN") {
- val rmClientSpy = spy(rmClient)
+ val rmClientSpy = spy[AMRMClient[ContainerRequest]](rmClient)
val maxExecutors = 11
val (handler, _) = createAllocator(
@@ -763,7 +763,7 @@ class YarnAllocatorSuite extends SparkFunSuite with
Matchers {
test("Test YARN container decommissioning") {
val rmClient: AMRMClient[ContainerRequest] = AMRMClient.createAMRMClient()
- val rmClientSpy = spy(rmClient)
+ val rmClientSpy = spy[AMRMClient[ContainerRequest]](rmClient)
val allocateResponse = mock(classOf[AllocateResponse])
val (handler, sparkConfClone) = createAllocator(3, rmClientSpy)
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisExternalCatalogSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisExternalCatalogSuite.scala
index df99cd851cc..95233b13438 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisExternalCatalogSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisExternalCatalogSuite.scala
@@ -48,7 +48,7 @@ class AnalysisExternalCatalogSuite extends AnalysisTest with
Matchers {
test("query builtin functions don't call the external catalog") {
withTempDir { tempDir =>
val inMemoryCatalog = new InMemoryCatalog
- val catalog = spy(inMemoryCatalog)
+ val catalog = spy[InMemoryCatalog](inMemoryCatalog)
val analyzer = getAnalyzer(catalog, tempDir)
reset(catalog)
val testRelation = LocalRelation(AttributeReference("a", IntegerType,
nullable = true)())
@@ -63,7 +63,7 @@ class AnalysisExternalCatalogSuite extends AnalysisTest with
Matchers {
test("check the existence of builtin functions don't call the external
catalog") {
withTempDir { tempDir =>
val inMemoryCatalog = new InMemoryCatalog
- val externCatalog = spy(inMemoryCatalog)
+ val externCatalog = spy[InMemoryCatalog](inMemoryCatalog)
val catalog = new SessionCatalog(externCatalog, FunctionRegistry.builtin)
catalog.createDatabase(
CatalogDatabase("default", "", new URI(tempDir.toString), Map.empty),
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TableLookupCacheSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TableLookupCacheSuite.scala
index 7d6ad3bc609..399799983fd 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TableLookupCacheSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TableLookupCacheSuite.scala
@@ -74,7 +74,7 @@ class TableLookupCacheSuite extends AnalysisTest with
Matchers {
test("table lookups to external catalog are cached") {
withTempDir { tempDir =>
val inMemoryCatalog = new InMemoryCatalog
- val catalog = spy(inMemoryCatalog)
+ val catalog = spy[InMemoryCatalog](inMemoryCatalog)
val analyzer = getAnalyzer(catalog, tempDir)
reset(catalog)
analyzer.execute(table("t1").join(table("t1")).join(table("t1")))
@@ -85,7 +85,7 @@ class TableLookupCacheSuite extends AnalysisTest with
Matchers {
test("table lookups via nested views are cached") {
withTempDir { tempDir =>
val inMemoryCatalog = new InMemoryCatalog
- val catalog = spy(inMemoryCatalog)
+ val catalog = spy[InMemoryCatalog](inMemoryCatalog)
val analyzer = getAnalyzer(catalog, tempDir)
val viewDef = CatalogTable(
TableIdentifier("view", Some("default")),
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala
index 062814e58b9..2c242965339 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala
@@ -46,7 +46,7 @@ class JoinSuite extends QueryTest with SharedSparkSession
with AdaptiveSparkPlan
// test case
plan.foreachUp {
case s: SortExec =>
- val sortExec = spy(s)
+ val sortExec = spy[SortExec](s)
verify(sortExec, atLeastOnce).cleanupResources()
verify(sortExec.rowSorter, atLeastOnce).cleanupResources()
case _ =>
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
index c37722133cb..377596466db 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
@@ -19,7 +19,7 @@ package org.apache.spark.sql.errors
import java.io.{File, IOException}
import java.net.{URI, URL}
-import java.sql.{Connection, Driver, DriverManager, PreparedStatement,
ResultSet, ResultSetMetaData}
+import java.sql.{Connection, DatabaseMetaData, Driver, DriverManager,
PreparedStatement, ResultSet, ResultSetMetaData}
import java.util.{Locale, Properties, ServiceConfigurationError}
import org.apache.hadoop.fs.{LocalFileSystem, Path}
@@ -743,8 +743,8 @@ class QueryExecutionErrorsSuite
val driver: Driver = DriverRegistry.get(driverClass)
val connection = ConnectionProvider.create(
driver, options.parameters, options.connectionProviderName)
- val spyConnection = spy(connection)
- val spyMetaData = spy(connection.getMetaData)
+ val spyConnection = spy[Connection](connection)
+ val spyMetaData = spy[DatabaseMetaData](connection.getMetaData)
when(spyConnection.getMetaData).thenReturn(spyMetaData)
when(spyMetaData.supportsTransactions()).thenReturn(false)
diff --git
a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperationSuite.scala
b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperationSuite.scala
index b61c91f3109..d085f596397 100644
---
a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperationSuite.scala
+++
b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperationSuite.scala
@@ -78,7 +78,7 @@ class SparkExecuteStatementOperationSuite extends
SparkFunSuite with SharedSpark
HiveThriftServer2.eventManager =
mock(classOf[HiveThriftServer2EventManager])
- val spySqlContext = spy(sqlContext)
+ val spySqlContext = spy[SQLContext](sqlContext)
// When cancel() is called on the operation, cleanup causes an exception
to be thrown inside
// of execute(). This should not cause the state to become ERROR. The
exception here will be
diff --git
a/streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala
b/streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala
index 7bdb8bd9a02..ada6a9a4cb6 100644
---
a/streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala
+++
b/streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala
@@ -139,7 +139,7 @@ class ReceivedBlockTrackerSuite extends SparkFunSuite with
BeforeAndAfter with M
}
test("block allocation to batch should not loose blocks from received
queue") {
- val tracker1 = spy(createTracker())
+ val tracker1 = spy[ReceivedBlockTracker](createTracker())
tracker1.isWriteAheadLogEnabled should be (true)
tracker1.getUnallocatedBlocks(streamId) shouldEqual Seq.empty
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
