dongjoon-hyun commented on code in PR #53242:
URL: https://github.com/apache/spark/pull/53242#discussion_r2570209912
##########
sql/core/src/test/scala/org/apache/spark/sql/VariantSuite.scala:
##########
@@ -820,7 +820,9 @@ class VariantSuite extends QueryTest with
SharedSparkSession with ExpressionEval
// The initial size of the buffer backing a cached dataframe column is
128KB.
// See `ColumnBuilder`.
val numKeys = 128 * 1024
- val keyIterator = (0 until numKeys).iterator
+ // We start in long range because the shredded writer writes int64 by
default which wouldn't
+ // match narrower binaries.
+ val keyIterator = (Int.MaxValue + 1L until Int.MaxValue + 1L +
numKeys).iterator
Review Comment:
Got it. Initially, I worried about the size because `numKeys` was 128k.
After I computed the actual values, I also realized that it doesn't increase
effectively.
```
val numKeys = 128 * 1024
val keyIterator1 = (0 until numKeys).iterator
val keyIterator2 = (Int.MaxValue + 1L until Int.MaxValue + 1L +
numKeys).iterator
val entries1 = Array.fill(numKeys)(s"""\"${keyIterator1.next()}\":
\"test\"""")
val entries2 = Array.fill(numKeys)(s"""\"${keyIterator2.next()}\":
\"test\"""")
val jsonStr1 = s"{${entries1.mkString(", ")}}"
val jsonStr2 = s"{${entries2.mkString(", ")}}"
scala> jsonStr1.length
val res1: Int = 2248186
scala> jsonStr2.length
val res2: Int = 2883584
```
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
