[
https://issues.apache.org/jira/browse/IMPALA-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496300#comment-16496300
]
Gabor Kaszab commented on IMPALA-7084:
--------------------------------------
Indeed. I pulled the latest code and this doesn't reproduce for me either.
Don't know what happened in the last 2 weeks but some of the changes must have
fixed this.
Thanks for looking into this issue [~fredyw]!
> Partition doesn't exist after attempting to ALTER partition location to
> non-existing path
> -----------------------------------------------------------------------------------------
>
> Key: IMPALA-7084
> URL: https://issues.apache.org/jira/browse/IMPALA-7084
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 2.12.0
> Reporter: Gabor Kaszab
> Priority: Major
> Labels: correctness, ramp-up
>
>
>
> {code:java}
> create table test (a int) partitioned by (b int);
> insert into test partition (b=1) values (1);
> // create another partition that points to a different location.
> alter table test add partition (b=2) location
> 'hdfs://localhost:20500/test-warehouse/test/b=2/';
> // setting the first partition to a non existing location. This fails as
> expected. The error message is not exactly user friendly, though. Could have
> said "Invalid location or such".
> alter table test partition (b=1) set location
> 'hdfs://localhost:20500/test-warehouse/test/b=5/';
> Query: alter table test partition (b=1) set location
> 'hdfs://localhost:20500/test-warehouse/test/b=5/'
> ERROR: TableLoadingException: Failed to load metadata for table: default.test
> CAUSED BY: NullPointerException: null
> // Setting the first partition to an existing location. This surprisingly
> fails as the partitions doesn't exist.
> alter table test partition (b=1) set location
> 'hdfs://localhost:20500/test-warehouse/test/b=2/';
> Query: alter table test partition (b=1) set location
> 'hdfs://localhost:20500/test-warehouse/test/b=1/'
> ERROR: PartitionNotFoundException: Partition not found:
> TPartitionKeyValue(name:b, value:1)
> // However, show partition displays b=1 partition as well.
> show partitions test;
> Query: show partitions test
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------+
> | b | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format |
> Incremental stats | Location |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------+
> | 1 | -1 | 1 | 2B | NOT CACHED | NOT CACHED | TEXT | false |
> hdfs://localhost:20500/test-warehouse/test/b=1 |
> | 2 | -1 | 1 | 2B | NOT CACHED | NOT CACHED | TEXT | false |
> hdfs://localhost:20500/test-warehouse/test/b=2 |
> | Total | -1 | 2 | 4B | 0B | | | | |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------+
> // Invalidate metadata fixes this issue
> invalidate metadata test;
> // Now show partitions say that b=1 partition points to the non-existing
> location. (This is not a problem as the location would be
> // created one data is put into that partition again.)
> show partitions test;
> Query: show partitions test
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------
> --------------------------+
> | b | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format |
> Incremental stats | Location
> |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------
> --------------------------+
> | 1 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false |
> hdfs://localhost:2050
> 0/test-warehouse/test/b=5 |
> | 2 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false |
> hdfs://localhost:2050
> 0/test-warehouse/test/b=2 |
> | Total | -1 | 0 | 0B | 0B | | | |
> |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------
> --------------------------+
> // After the invalidate metadata the partition is again visilbe for ALTER
> TABLE partition location commands.
> alter table test partition (b=1) set location
> 'hdfs://localhost:20500/test-warehouse/test/b=2/';
> Query: alter table test partition (b=1) set location
> 'hdfs://localhost:20500/test-warehouse/test/b=2/'
> +--------------------------------------------------------+
> | summary |
> +--------------------------------------------------------+
> | New location has been set for the specified partition. |
> +--------------------------------------------------------+
> // {code}
>
> My expectation here would be that in case the first alter partition location
> command fails due to non-existing location then no state change should be
> done in the background. In addition seeing an existing partition as a
> non-existing one after a failed attempt to change some metadata is also
> undesired.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
