zhaomin1423 commented on PR #43439:
URL: https://github.com/apache/spark/pull/43439#issuecomment-1779538121
> > > What I am confused about is, how should we check the effect after gc
is triggered
> >
> >
> > Construct a case where `SparkResult` won't be manually closed, and check
if after registering `cleanable`, `Cleaner.impl.phantomCleanableList` is not
empty, but eventually `phantomCleanableList` is empty? This may require adding
some code to get `phantomCleanableList` through reflection, just a suggestion,
not sure if it's really feasible.
>
> Thanks, I will try it and if it doesn't work. I will verify manually by
adding logs.
I did a manual test, there are two steps.
1. org.apache.spark.sql.connect.client.SparkResultCloseable#close add
```println("The resource of spark result closed.")```
```
override def close(): Unit = {
resultMap.values.foreach(_._2.foreach(_.close()))
responses.close()
// scalastyle:off println
println("The resource of spark result closed.")
}
```
2. start connect server and use example below to test. if ```The resource of
spark result closed.``` can print to console, it can work well.
```
object ConnectClientTest {
def main(args: Array[String]): Unit = {
val spark = SparkSession.builder
.remote("sc://localhost:15002")
.getOrCreate()
var sparkResult = spark.range(0, 10, 1, 10)
.collectResult()
// scalastyle:off println
sparkResult = null
for (_ <- 0 until 10) {
System.gc()
Thread.sleep(100L)
}
spark.stop()
}
}
```
Test result:
```The resource of spark result closed.``` can print to console, so the
org.apache.spark.sql.connect.client.SparkResultCloseable work well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
