JingsongLi commented on code in PR #975:
URL: https://github.com/apache/paimon-rust/pull/975#discussion_r4176927344
##########
crates/paimon/src/spec/schema.rs:
##########
@@ -360,6 +360,26 @@ impl TableSchema {
let field =
DataField::new(id, name.to_string(),
data_type).with_description(comment);
insert_field_with_move(&mut fields, field,
column_move.as_ref(), full_name)?;
+ // A CSV format table reads existing files positionally (no
+ // per-column header mapping), so a new column that lands
anywhere
+ // but last shifts every later physical column: the
permissive
+ // reader would pad/truncate against the wrong positions
and
+ // silently mis-assign old rows. Appending (the trailing
position)
+ // is safe — old files just pad the new column with null.
Reject
+ // the position-shifting case, symmetric with the
DropColumn guard.
+ {
+ let core_options =
CoreOptions::new(&new_schema.options);
+ if core_options.is_format_table()
+ && core_options.file_format() == "csv"
+ && field_index(&fields, name) != Some(fields.len()
- 1)
Review Comment:
[P2] Apply CSV tail checks to non-partition fields
This checks the last logical field, while FormatTableRead/Writer exclude
partition columns from the physical CSV schema (as [Java
FormatReadBuilder](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/table/format/FormatReadBuilder.java)
does). A real filesystem CSV table `(id, label, pt)` partitioned by pt reads
`pt=x/old.csv` containing `1,old` correctly, but adding `extra AFTER label` is
rejected here, even though its physical layout is the safe trailing append
`(id,label,extra)`. The corresponding new DropColumn check also rejects
dropping label, the last physical CSV column. Both catalog operations fail with
the physical-column-shift error in the probe, although neither shifts an
existing physical column. Compare positions among non-partition data fields in
both guards so partitioned CSV tables retain these supported safe changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
