————— 2022-5-17 ————— Ali eric 09:27 @peacewong@WDS Peace, you have encountered a large sql execution, but the ec resources given are few, and finally oom, the process on yarn hangs, and the job task also reports oom, but ec has been busy and stuck, Other tasks cannot be submitted, nor do they occupy resources. I see that it finally exited here, but the ec process is still there. Have you ever met?
peacewong@WDS 09:52 Take a look at the jstack information peacewong@WDS 09:53 1.1.1 A similar problem has been solved, and a PR will be issued later peacewong@WDS 10:18 @李eric You can compare this, the trigger conditions are similar, the problem is that this ec may need to be cleaned up manually, and will not cause other problems: https://github.com/apache/incubator-linkis/pull/1780 Arieric 10:21 receive [clasped fist] peacewong@WDS 10:31 @weizhong-xiachen With the help of this PR review, after stress testing, it was found that the performance of replace into is better https://github.com/apache/incubator-linkis/pull/2122 peacewong@WDS 11:13 @wForget-iqiyi-wangzhen @weizhong-xiachen help revie to download this PR and merge the latest code of dev-1.1.1 branch to master: https://github.com/apache/incubator-linkis/pull/2123 W 11:14 good [strong] ————— 2022-5-17 ————— 阿黎eric 09:27 @peacewong@WDS 和平，你们有遇到过一个大sql执行，但是给的ec资源少，最后oom了，yarn上的进程挂了，job任务也报出了oom，但是ec 一直繁忙卡住了，别的任务提交不进来，自己也不挂占着资源。我看这里最后是退出了，但是ec的进程还在。你们有遇到过吗？ peacewong@WDS 09:52 看看jstack信息下 peacewong@WDS 09:53 1.1.1有解决过一个类似的问题，晚点发下PR peacewong@WDS 10:18 @阿黎eric 可以对比下这个，触发条件比较相似，导致的问题就是这个ec可能需要手动清理，不会导致其他问题：https://github.com/apache/incubator-linkis/pull/1780 阿黎eric 10:21 收到[抱拳] peacewong@WDS 10:31 @微众-夏晨 这个PR帮忙review下，经过压测发现还是replace into性能更好https://github.com/apache/incubator-linkis/pull/2122 peacewong@WDS 11:13 @wForget-爱奇艺-王震 @微众-夏晨 帮忙revie下这个PR，合并dev-1.1.1分支最新代码到master：https://github.com/apache/incubator-linkis/pull/2123 W 11:14 好的 [强] --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@linkis.apache.org For additional commands, e-mail: dev-h...@linkis.apache.org
