scovich commented on PR #8166: URL: https://github.com/apache/arrow-rs/pull/8166#issuecomment-3265914456
> > I'm pretty sure the actual solution will be to merge the new variant row builder infrastructure in this PR, and then rework `cast_to_variant` to use it. That way, the row builder's constructor can do any column-level transformations that might be needed, before row-oriented visiting begins. > > Yes, this sounds like a very plausible approach > > > In other words, this PR is probably the most important (= biggest bottleneck) variant PR currently open. > > This is my feeling too. @carpecodeum if you don't have time to work on this PR in the next day or two, perhaps @scovich or @liamzwbao could open a new PR (starting with the code from this PR) that we can finish up? > > > NOTE: Even tho `variant_get` and `cast_to_variant` would both use the new row builder infra, they are not doing the same thing. The former (when unshredding) needs to use a read-only metadata builder because the `metadata` column already exists; the latter would use a normal metadata builder. So `cast_to_variant` may actually be lower hanging fruit for converting "stuff" to binary variant, with `variant_get` support for the same landing later when we sort out the read-only metadata thing. > > 👍 UPDATE: I started doing some pathfinding on a row-oriented `cast_to_variant`, and it turns out _not_ to need this PR. The _approach_ is the same, but shredding needs variant-to-arrow row builders while the casting needs arrow-to-variant row builders. I'll try to post a PR soon that takes a stab at the latter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
