Hi Laszlo,
As Jack mentioned, this behavior happened to v1 format.
If you want to stay with v1 format, a short term workaround is to add back
this field with a different name, e.g.
table.updateSpec().addField("ts_month_new", month("ts")).commit();
or rename the field before deleting it
table.updateSpec().renameField("ts_month", "ts_month_deleted").commit();
table.updateSpec().removeField("ts_month_deleted").commit();
I think this may be fixed by reviving the null transform if a newly added
partition field is identical or compatible with the previously dropped one.
I will take a look at it.
Jun
On Wed, Jun 9, 2021 at 8:27 AM Jack Ye <yezhao...@gmail.com> wrote:
> Hi Laszlo,
>
> I think this is expected behavior of format v1, where a dropped partition
> column is actually converted to a null transform instead of being actually
> dropped. This is because the partition spec is not versioned in v1.
>
> See
> https://github.com/apache/iceberg/blob/efaad975fc9bbeaa27392b333211e99ac444aaba/core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java#L222-L226
> for more details.
>
> -Jack Ye
>
> On Wed, Jun 9, 2021 at 6:54 AM Laszlo Pinter <lpin...@cloudera.com.invalid>
> wrote:
>
>> Hi Iceberg Devs,
>>
>> I'm using the Iceberg API to drop/add partition transforms, but recently
>> I've run into an issue.
>> When I try to add a new partition that previously existed, but it was
>> already dropped I get *Cannot use partition name more than once *error
>> message.
>>
>> Here is what I'm doing:
>>
>> table.updateSpec().addField(month("ts")).commit();
>> table.updateSpec().removeField("ts_month").commit();
>> table.updateSpec().addField(day("ts")).commit();
>> table.updateSpec().addField(month("ts")).commit();
>>
>> What is the correct way to implement this sort of partition evolution?
>>
>> Thanks,
>> Laszlo
>>
>
