GideonPotok commented on PR #46670:
URL: https://github.com/apache/spark/pull/46670#issuecomment-2123283997
@uros-db adding the map binaryValues to Mode doesn't persist the mappings as
desired. It looks to me like the general pattern for accumulating anything in
spark aggregation functions is by putting it into an InternalRow and having
merge and update take internal row as parameter.
Is there another way you advise? Do I use a class that extends from
InternalRow, which would include both the OpenHashMap[AnyRef, Long] as well as
our Map[String, String] ? Let me know what the right pattern is because
obviously having that element of state be a member of Mode implementation is
not the standard way/ doesn't seem to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
