zhuxiangyi commented on code in PR #8943:
URL: https://github.com/apache/paimon/pull/8943#discussion_r3705731330
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/data/SparkInternalRow.scala:
##########
@@ -33,6 +34,14 @@ abstract class SparkInternalRow extends InternalRow {
object SparkInternalRow {
+ private val blobFieldsCache: LoadingCache[RowType, Set[Int]] =
Review Comment:
Good suggestion. Since RowType is immutable and already maintains transient
lazy caches for derived field mappings, caching the BLOB field indices there
would give the cache the same lifecycle as the RowType and avoid the global
weak-key cache in the Spark module.
I’ll update the PR accordingly and keep the cached representation
independent of Spark/Scala types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
