[
https://issues.apache.org/jira/browse/KYLIN-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-3963:
----------------------------
Fix Version/s: v3.1.0
> Provide a custom serializer for PercentileCounter
> -------------------------------------------------
>
> Key: KYLIN-3963
> URL: https://issues.apache.org/jira/browse/KYLIN-3963
> Project: Kylin
> Issue Type: Sub-task
> Components: Flink Engine
> Reporter: vinoyang
> Assignee: vinoyang
> Priority: Major
> Fix For: v3.1.0
>
>
> {{PercentileCounter}} contains an instance field of Type {{TDigest}} which is
> not serializable, so it is marked as {{transient}}. By default, it triggers a
> NPE when use Flink cube engine:
> {code:java}
> Caused by: org.apache.flink.client.program.ProgramInvocationException: Job
> failed. (JobID: 8077ff0e0ff01a26f62b1babe90a34b7) at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:475)
> at
> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer.execute(FlinkCubingByLayer.java:196)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 18 more Caused by:
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
> at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
> ... 23 more
> Caused by: java.lang.NullPointerException at
> org.apache.kylin.measure.percentile.PercentileCounter.writeRegisters(PercentileCounter.java:68)
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:70)
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:26)
> at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:76) at
> org.apache.kylin.measure.BufferedMeasureCodec.encode(BufferedMeasureCodec.java:93)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:234)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:218)
> at org.apache.flink.runtime.operators.MapDriver.run(MapDriver.java:103) at
> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503) at
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) at
> java.lang.Thread.run(Thread.java:745)
> {code}
> So we should provide a customized serializer for {{PercentileCounter}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
