talmacschen-arch commented on issue #1830:
URL: https://github.com/apache/cloudberry/issues/1830#issuecomment-4932317524
@adnanhamdussalam @tuhaihe — following up, and I want to correct part of my
earlier analysis based on a closer read of your own logs.
**Your logs show every expected connection did establish.** `client-numbers
4,4,4,4,3,3,3,3` means the Seg1 host (segs 0–3) expects 16 inbound connections
and the Seg2 host (segs 4–7) expects 12. Your logs show exactly 16
`ConcurrentServer accept a connection` lines on TIM-CB-Seg1 and 12 on
TIM-CB-Seg2 — a complete set — and ~334 MB actually transferred. So my earlier
guess (a missing source→dest link / firewall) does **not** fit your data; no
connection was missing.
**What actually ends it:** the timestamps are `13:46:18 → 13:56:18`, exactly
**600 seconds**. cbcopy's receive-side helper sets a 600 s deadline on its
*listener* (`helper/server.go`), and its accept loop keeps running for the
whole life of the transfer — it never stops after the expected clients connect.
When that absolute deadline expires, `Accept()` returns `i/o timeout`, which
sets the server error and tears the helper down **while data is still streaming
on the already-established connections**. The source side then reports
`connection reset by peer` / `Broken pipe`, and the destination QE reads a
truncated row → `missing data for column (22P04)`.
So this behaves as a **hard ~10-minute cap on how long a single table's data
transfer may run**, independent of connectivity: any table whose data takes
longer than 600 s to move gets torn down. That explains why small tables
succeed and large ones fail, and why your 115 GB table aborted with only 334 MB
moved.
**To confirm and quantify, could you share, per affected table:**
1. **Row count** — `SELECT count(*) FROM <schema.table>;`
2. **On-disk size** — `SELECT
pg_size_pretty(pg_total_relation_size('<schema.table>'));` (you gave 115 GB for
`tim_expansion_crm_btntiid`; its row count would complete the picture.)
3. Does it **always fail right around the 10-minute mark**, and do smaller
tables succeed?
4. If possible, sample the destination table size every ~30 s while it runs
— is it **still growing** at the 10-minute mark (slow-but-progressing) or
**flat** (stalled)?
If the pattern holds (transfer time > 600 s ⇒ 22P04 at ~10 min), the fix
belongs in the cbcopy helper: the listener deadline should bound only the
initial accept phase, not the whole transfer. I can move the fix discussion to
the cbcopy tracker (https://github.com/cloudberry-contrib/cbcopy/issues).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
