tustvold commented on issue #506: URL: https://github.com/apache/arrow-rs/issues/506#issuecomment-870495007
> @tustvold This sounds pretty interesting. Would the idea be to add potentially duplicated / unused values in MutableArrayData and then rewrite the data on freeze? The idea I had was be to defer creation of the dictionary ArrayData until freeze, as opposed to in the constructor, but otherwise to keep the extend generation the same. Freeze would then scan through the generated keys in the `_MutableArrayData` to compute the values to include in the final dictionary, build this new dictionary, and then rewrite the keys. I realise as I write this that `_MutableArrayData` may not have the necessary API to actually mutate data in place, but I don't see a compelling reason that it couldn't? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
