felipepessoto commented on PR #12215: URL: https://github.com/apache/gluten/pull/12215#issuecomment-5301177606
@zhztheplayer I couldn't reproduce it locally — including the real `*MergeIntoExtendedSyntax*DVs*` suites from Delta v4.2.0 on Spark 4.1 (200 tests, incl. `...DVsPredPushOnSuite`). The native `_metadata.row_index` values always matched vanilla exactly. That said, I don't think `9223372036854775807` was ever written or read as a row ID. Delta's own `RoaringBitmapArray.add` enforces the identical bound (`MAX_REPRESENTABLE_VALUE = 9223372030412324864`), so vanilla Delta would abort the same way if that value ever reached the aggregator. And the sibling signature we quarantined for the same root cause is `Delta bitmap row index cannot be negative: -6254810385378525259` — arbitrary, not a sentinel. So it looks like garbage reaching the aggregator's input column, not a real row index. Since it only shows up under the full Delta suite, I opened #12783 to attach the offending column (encoding, nulls, value distribution) to the error, which should tell us whether it's a real row-index column with one corrupt slot or the wrong column entirely. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
