Rachelint commented on issue #6906: URL: https://github.com/apache/datafusion/issues/6906#issuecomment-2356370758
Maybe it is not a good way, but can we > @Rachelint your implicit idea of using `Vec<String>` to store the state I think is actually quite interesting and maybe we should try that one first: > > It would at least avoid calling `Array::slice` and likely be better than using `GroupsAccumulatorAdapter`, even if we could improve it later with more explciit memory management 🤔 🤔 Can we still use the string view like approach to store states https://github.com/apache/datafusion/issues/6906#issuecomment-2355604402, but for the uninlined state, we use a single `String` to store it? And when we output them to `StringViewArray`, we make convert this single `String` to a tiny buffer. For example, if all states are uninlined(len > 12), the output `StringViewArray` may be like: ``` row1: view1 buffer1(with only one string) row2: view2 buffer2(with only one string) ... rown: viewn buffern ``` I am not familiar enough with `StringViewArray`, is it ok to do that? And will it lead to a extremely bad performance? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
