Cq-study commented on issue #639: URL: https://github.com/apache/doris-flink-connector/issues/639#issuecomment-5084113235
After digging into this, the zeros on the overview page are expected Flink behavior rather than a connector bug: - The overview "Bytes/Records Received/Sent" numbers only count data exchanged **between tasks** (over the network). A CdcTools job (`CDC source -> ParsingProcessFunction -> side outputs -> Doris sinks`) is fully chainable, so the whole pipeline runs as a single task and there is no inter-task edge to count. - Additionally, sources never report "Records Received" and sinks never report "Records Sent" on the overview, because external-system I/O is not part of these task-level metrics. What you can do today: 1. Check the sink operator's **Metrics tab**: the connector reports per-table counters such as `<table>_totalFlushLoadedRows` / `<table>_totalFlushLoadBytes`. 2. For debugging, disable operator chaining (`pipeline.operator-chaining: false`) to make the internal edges visible (at a performance cost). I've also opened #681 which wires the standard FLIP-33 sink metrics `numRecordsSend` / `numBytesSend` into the Doris sink, so standard dashboards and the metrics tab can observe sink throughput without knowing the connector-specific counter names. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
