[
https://issues.apache.org/jira/browse/FLINK-16627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255180#comment-17255180
]
Jark Wu commented on FLINK-16627:
---------------------------------
Hi [~jackray], are you still working on this issue?
I agree with [~nilerzhou] that a new config option key style has been
introduced in FLIP-122, we may need to come up with a new key for this option.
{{'json.encode.ignore-null-fields'=false/true}} sounds good to me. As an
alternative, {{'json.encode.include-null-fields'=true/false}} is also fine to
me.
> Support only generate non-null values when serializing into JSON
> ----------------------------------------------------------------
>
> Key: FLINK-16627
> URL: https://issues.apache.org/jira/browse/FLINK-16627
> Project: Flink
> Issue Type: New Feature
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Planner
> Affects Versions: 1.10.0
> Reporter: jackray wang
> Assignee: jackray wang
> Priority: Major
> Fix For: 1.13.0
>
>
> {code:java}
> //sql
> CREATE TABLE sink_kafka ( subtype STRING , svt STRING ) WITH (……）
> {code}
>
> {code:java}
> //sql
> CREATE TABLE source_kafka ( subtype STRING , svt STRING ) WITH (……）
> {code}
>
> {code:java}
> //scala udf
> class ScalaUpper extends ScalarFunction {
> def eval(str: String) : String= {
> if(str == null){
> return ""
> }else{
> return str
> }
> }
>
> }
> btenv.registerFunction("scala_upper", new ScalaUpper())
> {code}
>
> {code:java}
> //sql
> insert into sink_kafka select subtype, scala_upper(svt) from source_kafka
> {code}
>
>
> ----
> Sometimes the svt's value is null, inert into kafkas json like
> \{"subtype":"qin","svt":null}
> If the amount of data is small, it is acceptable，but we process 10TB of data
> every day, and there may be many nulls in the json, which affects the
> efficiency. If you can add a parameter to remove the null key when defining a
> sinktable, the performance will be greatly improved
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
