andygrove opened a new issue, #7725:
URL: https://github.com/apache/datafusion-comet/issues/7725

   ### Describe the bug
   
   With `spark.comet.expression.RegExpExtract.allowIncompatible` (or 
`RegExpExtractAll`) enabled, native `regexp_extract` and `regexp_extract_all` 
can still fail on an out-of-range `idx` where Spark succeeds. This is the one 
case left after #7710.
   
   Spark checks `idx` only on a row where the pattern matches, in 
`RegExpExtractBase.checkGroupIndex`. #7710 makes the native functions do the 
same by scanning the batch with `Regex::is_match` and erroring only if a 
non-null row matches.
   
   A scalar subject is different. Comet scalar functions do not receive the 
batch row count, so a literal subject that matches the pattern errors even when 
the batch has zero rows. Spark evaluates no rows and returns an empty result.
   
   Reaching it takes a literal subject that constant folding did not remove, 
and a batch with no rows. It is rare.
   
   ### Steps to reproduce
   
   Needs a non-folded literal subject evaluated over an empty batch, with an 
out-of-range `idx`:
   
   ```sql
   -- spark.comet.expression.RegExpExtract.allowIncompatible=true
   SELECT regexp_extract('a1', '\\d+', 1) FROM t WHERE false;
   -- Spark: no rows, no error
   -- Comet: "Expects group index between 0 and 0, but got 1."
   ```
   
   I have not run this against Comet. It comes from the description of #7710 
and from reading `parse_args` and `any_row_matches` in 
`native/spark-expr/src/string_funcs/regexp_extract_common.rs`.
   
   ### Expected behavior
   
   Return no error when the batch has no rows. Fixing it means giving Comet 
scalar functions the batch row count (`number_rows`), which touches how every 
scalar function is called. The same missing row count may affect other scalar 
functions that decide something per row.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to