kylebarron commented on PR #35780: URL: https://github.com/apache/arrow/pull/35780#issuecomment-1837599573
I do feel that supporting the full spec (i.e. _having_ a `LargeUtf8` type) and supporting massive strings larger than `Number.MAX_SAFE_INTEGER` are two separate objectives. I personally feel that it's fine to use `number` for indexing (with some safety checks to avoid overflows) and to instruct people to use Vectors if they need an offset value larger than `Number.MAX_SAFE_INTEGER`. I think "a new data structure that chunks" is a lot of added complexity when `Vector` is _already_ a data structure that chunks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
