Hi Kang-Sen,
Have you check your hive table, whether there is any dirty data？ By the way, empty data in your first dimension may also cause such an exception. It has been fixed in 2.6.1 https://issues.apache.org/jira/browse/KYLIN-3828 -- Best regards, Ni Chunen / George At 2019-04-09 00:04:40, "Lu, Kang-Sen" <[email protected]> wrote: Hi, Chunen: Thanks for your reply. I am puzzled by the fact that based on the same data model, I created two cubes, one for computing TOPN metric, and the other for all other aggregation. The reason I separate the TOPN cube creation from the other normal cube is because the TOPN is related to a dimension with high cardinality like SUBSCRIBER_ID. The same fact table is used to build cuboids for both cube spec. I don’t have problem when building the TOPN cube, with spark engine. But when I build a cube with spark engine for the normal cube, I had this “\N” format exception. In addition, if I build this normal cube with MR engine, there is no format exception. Does it make sense to you? Kang-sen From: [email protected] <[email protected]> On Behalf Of nichunen Sent: Monday, April 08, 2019 11:35 AM To: [email protected] Subject: Re:question about kylin cube build failure NOTICE: This email was received from an EXTERNAL sender Hi Kang-Sen, It looks like there is a "\n" in your source data of a column with double type. -- Best regards, Ni Chunen / George At 2019-04-08 23:10:05, "Lu, Kang-Sen" <[email protected]> wrote: I am running kylin 2.5.1. When I am building a cube with spark engine, I got the following error at “#4 Step Name: Extract Fact Table Distinct Columns”. The log shows the following exception: 2019-04-08 12:59:10,375 WARN scheduler.TaskSetManager: Lost task 5.0 in stage 0.0 (TID 0, hadoop9, executor 1): java.lang.NumberFormatException: For input string: "\N" at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) at java.lang.Double.parseDouble(Double.java:538) at org.apache.kylin.engine.mr.steps.SelfDefineSortableKey.init(SelfDefineSortableKey.java:57) at org.apache.kylin.engine.mr.steps.SelfDefineSortableKey.init(SelfDefineSortableKey.java:66) at org.apache.kylin.engine.spark.SparkFactDistinct$FlatOutputFucntion.addFieldValue(SparkFactDistinct.java:444) at org.apache.kylin.engine.spark.SparkFactDistinct$FlatOutputFucntion.call(SparkFactDistinct.java:315) at org.apache.kylin.engine.spark.SparkFactDistinct$FlatOutputFucntion.call(SparkFactDistinct.java:226) at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186) at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186) at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:797) at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:797) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Anybody saw this same problem? Thanks. Kang-sen Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.
