talmacschen-arch commented on issue #1830:
URL: https://github.com/apache/cloudberry/issues/1830#issuecomment-4788798189
Hi @adnanhamdussalam, thanks for the report. A few things to help narrow
this down.
`SQLSTATE 22P04 / missing data for column "data_tec_asset"` is raised on the
**destination side** (`COPY ... FROM`): a CSV row arriving at the target had
fewer fields than the table has, so parsing ran out of input before reaching
`data_tec_asset`. The `seg2 slice1` in the message means this table is being
transferred on the **segments** (large table / copy-on-segment), not on the
coordinator. There are two main causes:
**1. The data transfer was interrupted mid-stream.** This is the most common
source of this exact message — see the troubleshooting section in the cbcopy
README. If the destination `cbcopy_helper` exits abruptly or the connection
breaks, the QE reads a partial row from the pipe and reports `missing data for
column`. Typical triggers:
- Segment-to-segment network/firewall blocking. A tell-tale sign is **small
tables succeed but large tables fail**, because large tables go
segment-to-segment.
- A manually cancelled query, or a helper that crashed/was killed.
- This case is usually accompanied by a `could not write to copy program:
Broken pipe` error on the **source** side.
**2. A column-count / column-order mismatch between the source and
destination tables.** cbcopy transfers via positional CSV with no explicit
column list, so the source and destination must have the same number of columns
in the same order. If the destination `staging.tmp_base_deduped` was
pre-created (e.g. by an ETL job) with an extra column or a different order,
every row fails at the same column. This case is **deterministic** — it fails
at the same column on every run, regardless of table size.
To tell these apart, could you share:
1. The full `cbcopy` command line (copy mode, `--truncate`, push/pull, etc.).
2. Whether it fails on the **same column every time**, and whether **only
large tables** fail while small ones succeed.
3. Whether the **source** log shows `could not write to copy program: Broken
pipe` around the same time.
4. Whether `staging.tmp_base_deduped` was created by cbcopy or pre-existed
at the destination.
5. `\d staging.tmp_base_deduped` on **both** source and destination (column
count and the position of `data_tec_asset`), plus:
```sql
SELECT attname, attnum, attisdropped FROM pg_attribute
WHERE attrelid = 'staging.tmp_base_deduped'::regclass ORDER BY attnum;
```
Also, a friendly heads-up: cbcopy has its own issue tracker at
https://github.com/cloudberry-contrib/cbcopy/issues — filing cbcopy-specific
issues there will get them in front of the cbcopy maintainers faster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
