Hi Iceberg Community, I'd like to propose extending Iceberg's type promotion rules to support DECIMAL type evolution with scale changes, aligning with the SQL:2011 standard.
*Current Limitation* Currently, Iceberg only supports DECIMAL type promotion when: - Scale remains the same - Precision can be increased This means DECIMAL(10,2) can evolve to DECIMAL(12,2), but not to DECIMAL(12,4). *Proposed Change* Allow DECIMAL type evolution when: 1. Target scale >= source scale 2. Target precision >= source precision 3. Integer part capacity is preserved: (target_precision - target_scale) >= (source_precision - source_scale) *Examples* With this change: - DECIMAL(10,2) → DECIMAL(12,4) ✓ (integer part: 8 → 8, scale: 2 → 4) - DECIMAL(10,2) → DECIMAL(15,5) ✓ (integer part: 8 → 10, scale: 2 → 5) - DECIMAL(10,2) → DECIMAL(10,4) ✗ (integer part: 8 → 6, would lose integer capacity) *Rationale* 1. SQL:2011 Compliance: This behavior aligns with SQL:2011 standard expectations 2. User Experience: Many users coming from traditional databases expect this type evolution to work 3. Data Safety: The proposed rules ensure no data loss - existing values can always be represented in the new type 4. Real-world Use Cases: Common scenarios like adding more decimal precision for currency calculations would be supported *Implementation* I've created a proof-of-concept implementation: https://github.com/apache/iceberg/issues/14037 *Questions for Discussion* 1. Should this be part of the spec v3, or wait for a future version? 2. Are there any backward compatibility concerns we should address? Looking forward to your feedback and thoughts on this proposal. Best regards, Minglei
