Hi Dylan, I think Rex encountered another issue, because he is using Kafka with Debezium format.
Hi Rex, If you can share the json data and the exception stack, that would be helpful! Besides, you can try to enable 'debezium-json.ignore-parse-errors' option [1] to skip the dirty data. Best, Jark [1]: https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/formats/debezium.html#debezium-json-ignore-parse-errors On Thu, 19 Nov 2020 at 21:13, Dylan Forciea <[email protected]> wrote: > Do you mean that the array contains values that are null, or that the > entire array itself is null? If it’s the latter, I have an issue written, > along with a PR to fix it that has been pending review [1]. > > > > Regards, > > Dylan Forciea > > > > [1] https://issues.apache.org/jira/browse/FLINK-19771 > > > > *From: *Danny Chan <[email protected]> > *Date: *Thursday, November 19, 2020 at 2:24 AM > *To: *Rex Fenley <[email protected]> > *Cc: *Flink ML <[email protected]> > *Subject: *Re: Filter Null in Array in SQL Connector > > > > Hi, Fenley ~ > > > > You are right, parsing nulls of ARRAY field is not supported now, i have > logged an issue [1] and would fix it soon ~ > > > > [1] https://issues.apache.org/jira/browse/FLINK-20234 > > > > Rex Fenley <[email protected]> 于2020年11月19日周四 下午2:51写道: > > Hi, > > > > I recently discovered some of our data has NULL values arriving in an > ARRAY<STRING> column. This column is being consumed by Flink via the Kafka > connector Debezium format. We seem to be receiving NullPointerExceptions > for when these NULL values in the arrays arrive which restarts the source > operator in a loop. > > > > Is there any way to not throw or to possibly filter out NULLs in an Array > of Strings in Flink? > > > > We're somewhat stuck on how to solve this problem, we'd like to be > defensive about this on Flink's side. > > > > Thanks! > > > > (P.S. The exception was not that informative, there may be room for > improvement in terms of a richer error message when this happens.) > > > -- > > *Rex Fenley* | Software Engineer - Mobile and Backend > > > > *Remind.com* <https://www.remind.com/> | BLOG <http://blog.remind.com/> > | FOLLOW US <https://twitter.com/remindhq> | *LIKE US > <https://www.facebook.com/remindhq>* > >
