uros-b commented on code in PR #56621:
URL: https://github.com/apache/spark/pull/56621#discussion_r3444962944
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/SizeInBytesOnlyStatsPlanVisitor.scala:
##########
@@ -74,7 +84,7 @@ object SizeInBytesOnlyStatsPlanVisitor extends
LogicalPlanVisitor[Statistics] {
override def visitExpand(p: Expand): Statistics = {
val sizeInBytes = visitUnaryNode(p).sizeInBytes * p.projections.length
- Statistics(sizeInBytes = sizeInBytes)
+ Statistics(sizeInBytes = sizeInBytes.min(MAX_SIZE_IN_BYTES))
Review Comment:
Nit: the cap is applied at `visitUnaryNode`, `default`, and `visitExpand`
individually; centralizing in one private helper would reduce the chance future
size-producing branches forget the bound.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
