Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16196#discussion_r91374827
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -221,8 +226,13 @@ object SizeEstimator extends Logging {
case _ =>
val classInfo = getClassInfo(cls)
state.size += alignSize(classInfo.shellSize)
- for (field <- classInfo.pointerFields) {
- state.enqueue(field.get(obj))
+ // avoid an iterator based for loop for performance
+ var index = 0
+ val fieldCount = classInfo.fieldOffsets.length
+ val us = Unsafe.instance
+ while (index < fieldCount) {
+ state.enqueue(us.getObject(obj,
classInfo.fieldOffsets(index).toLong))
--- End diff --
I understand avoiding reflection, but this is a dicier way to access fields
of an object. I don't have a specific reason this would fail but the fact that
it uses unsafe is riskier. Is this worth it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
