jobmanager异常

2020-08-16 Thread 18500348...@163.com 
请教大家一个问题：

flink1.8.0 on yarn 程序运行一段时间报如下错误，导致 The heartbeat of TaskManager with id 
container_1572430463280_50994_01_04 timed out. 最终程序重启。

各位有没有碰到类似的问题，有什么解决方式吗？

jobmanager.log

2020-08-17 02:53:21,593 ERROR akka.remote.Remoting  
- Association to [akka.tcp://flink@${HOSTNAME}:36968] with UID [19
99537927] irrecoverably failed. Quarantining address.
java.util.concurrent.TimeoutException: Remote system has been silent for too 
long. (more than 48.0 hours)
at 
akka.remote.ReliableDeliverySupervisor$$anonfun$idle$1.applyOrElse(Endpoint.scala:375)
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
at 
akka.remote.ReliableDeliverySupervisor.aroundReceive(Endpoint.scala:203)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at 
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at 
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at 
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)





18500348...@163.com

flink1.11 sql kafka 抽取事件时间

2020-07-15 Thread 18500348...@163.com 
大家好！

使用flink1.11 sql接入kafka ，format为csv
从eventTime字段中抽取事件时间
rowtime AS TO_TIMESTAMP(FROM_UNIXTIME(eventTime / 1000, '-MM-dd HH:mm:ss'))
eventTime可能存在脏数据（非13位的毫秒时间戳），设置了 'csv.ignore-parse-errors' = 'true', 
那么eventTime会被设置为null，此时会报一个异常：
Caused by: java.lang.RuntimeException: RowTime field should not be null, please 
convert it to a non-null long value.

有没有什么好的方式可以解决


祝好！



18500348...@163.com