cloud-fan commented on code in PR #58865:
URL: https://github.com/apache/spark/pull/58865#discussion_r4105578480
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala:
##########
@@ -99,6 +100,30 @@ case class FakeLeafPlan(child: LogicalPlan)
case class FakeCurryingProduct(x: Expression)(val y: Int)
class TreeNodeSuite extends SparkFunSuite with SQLHelper {
+ test("constructing unused Literals does not allocate eager cache state") {
+ val bean =
ManagementFactory.getThreadMXBean.asInstanceOf[com.sun.management.ThreadMXBean]
+ assert(bean.isThreadAllocatedMemorySupported)
+ bean.setThreadAllocatedMemoryEnabled(true)
+ val count = 100000
+ val literals = new Array[Literal](count)
+
+ var i = 0
+ while (i < count) {
+ Literal(1)
+ i += 1
+ }
+
+ val before = bean.getCurrentThreadAllocatedBytes
+ i = 0
+ while (i < count) {
+ literals(i) = Literal(1)
+ i += 1
Review Comment:
**Non-blocking (P2):** `literals` is never observed after this loop. Because
the array and its `Literal` graph do not escape, an optimizing JVM can
eliminate these stores and allocations, letting the test pass even if eager
cache allocation regresses. Please consume the populated array through an
observable, non-inlinable sink after measuring so the allocation graph remains
live.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala:
##########
@@ -118,8 +136,19 @@ abstract class TreeNode[BaseType <: TreeNode[BaseType]]
* A BitSet of tree patterns for this TreeNode and its subtree. If this
TreeNode and its
* subtree contains a pattern `P`, the corresponding bit for `P.id` is set
in this BitSet.
*/
- private val _treePatternBits = new BestEffortLazyVal[BitSet](() =>
getDefaultTreePatternBits)
- override def treePatternBits: BitSet = _treePatternBits()
+ @volatile private var treePatternBitsCacheValue: BitSet = _
+ override def treePatternBits: BitSet = {
+ var value = treePatternBitsCacheValue
+ if (value eq null) {
+ val computed = getDefaultTreePatternBits
Review Comment:
**Non-blocking (P2):** This publication path is now owned by `TreeNode`, so
`BestEffortLazyValSuite` can no longer exercise it. Please add deterministic
TreeNode-level concurrent-initialization coverage that verifies racing calls
return the same CAS-selected object and a later call reuses it; one
representative accessor is sufficient for the shared pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
