One more thing: Looking back on the previous discussion[1] (which Weston
pointed out in his earlier message), Jorge suggested that the old list
types might be deprecated in favor of view variants [2]. Others were
worried that it might undermine the perception that the Arrow format is
stable. I
Hello,
I think Sasha brings up a good point, that the advantages of this format
seem to be primarily about query processing. Other encodings like REE and
dictionary have space-saving advantages that justify them simply in terms
of space efficiency (although they have query processing advantages
The benefit of having a memory format that’s friendly to non-deterministic
order writes is unlocked by the transport and processing of the data being
agnostic to the physical order as much as possible.
Requiring a conversion could cancel out that benefit. But it can be a
provisory step for
