[
https://issues.apache.org/jira/browse/SPARK-16409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-16409:
------------------------------
Priority: Major (was: Critical)
We're missing the stack trace here? that's an important piece of info. Also
what is your data like that triggers this, if possible?
> regexp_extract with optional groups causes NPE
> ----------------------------------------------
>
> Key: SPARK-16409
> URL: https://issues.apache.org/jira/browse/SPARK-16409
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.0.0
> Reporter: Max Moroz
>
> df.select(F.regexp_extract('s', r'(a+)(b)?(c)', 2)).collect()
> causes NPE. Worse, in a large program it doesn't cause NPE instantly; it
> actually works fine, until some unpredictable (and inconsistent) moment in
> the future when (presumably) the invalid memory access occurs, and then it
> fails. For this reason, it took several hours to debug this.
> Suggestion: either fill the group with null; or raise exception immediately
> after examining the argument with a message that optional groups are not
> allowed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
