Re: Flink cdc 多表关联处理延迟很大

Sun, 15 Nov 2020 18:05:53 -0800 

select 
    ri.sub_clazz_number, 
    prcrs.rounds, 
    count(*) as num 
from 
    subclazz gs 
JOIN 
    (SELECT gce.number, min( gce.extension_value ) AS grade FROM 
course_extension gce WHERE gce.isdel = 0 AND gce.extension_type = 4 GROUP BY 
gce.number) AS temp 
ON 
    temp.number = gs.course_number AND temp.grade>30 
JOIN 
    right_info ri
ON  
    gs.number = ri.sub_clazz_number 
join
    wide_subclazz ws  
on
    ws.number = ri.sub_clazz_number 
join 
    course gc 
on 
    gc.number = ws.course_number and gc.course_category_id in (30,40)  
left join 
    performance_regular_can_renewal_sign prcrs 
on prcrs.order_number = ri.order_number and    prcrs.rounds in (1,2) 
where ri.is_del = 0 and ri.end_idx = -1 and prcrs.rounds is not null 
and not exists (select 1 from internal_staff gis where gis.user_id = ri.user_id)
and not exists (select 1 from clazz_extension ce where ws.clazz_number = 
ce.number  
    and ce.extension_type = 3 and ce.isdel = 0  
    and ce.extension_value in (1,3,4,7,8,11))  
group by ri.sub_clazz_number, prcrs.rounds 
Sql代码是这样的。
瓶颈在所有的join节点上，每次的checkpoint无法完成的节点都是join节点。

> 在 2020年11月14日，下午5:53，Jark Wu <[email protected]> 写道：
> 
> 能展示下你的代码吗？是用的维表关联的语法 （FOR SYSTEM TIME AS OF）？
> 需要明确下，到底是什么节点慢了。
> 
> On Fri, 13 Nov 2020 at 19:02, 丁浩浩 <[email protected]> wrote:
> 
>> 我用flink cdc 对mysql的表进行关联查询，发现flink只能两张表关联之后再跟下一张表关联，导致最后落库的延迟非常大。
>> 有没有比较好的优化方案能缓解这样的问题？

回复