Hi guys I’m trying to do an ALTER TABLE for a Kudu range partitioned table. The ALTER TABLE itself works fine, my issue is that most of the time I get a delay of about 5min until the “show range partitions xyz” shows the current/updated range partitions and we need this show command to automatically update the table ranges and cut the range boundaries...
Sometimes the delay is there and sometimes not - at least after a restart of
impala. I’ve tried several things, removed the kudu table, dropped the table in
impala and re-added it plus other things, but I don’t get a reproducible
result. sometimes it’s fine, but most of the time it’s not.
If the issue persists, the two following commands doesn’t help to get the table
updated: `REFRESH` and `INVALIDATE METADAT`
We have 33 Impala nodes.
Any other ideas to solve the issue? Or is it well known. Thanks in advance
Josef
Example:
##################################################
## Kudu Partition Schema:
##################################################
HASH (flowEndDate, uniqueID) PARTITIONS 16,
RANGE (flowEndDate) (
PARTITION VALUES < 1537860190,
PARTITION 1537860190 <= VALUES < 1537863790,
PARTITION 1537863790 <= VALUES < 1537867390,
PARTITION 1537867390 <= VALUES < 1537870990,
PARTITION 1537870990 <= VALUES < 1537874590,
PARTITION 1537874590 <= VALUES < 1537878190,
PARTITION VALUES >= 1537878190
)
##################################################
## show partitions
##################################################
[:21000] > show range partitions test_sql_drop;
Query: show range partitions test_sql_drop
+-----------------------------------+
| RANGE (flowenddate) |
+-----------------------------------+
| VALUES < 1537860190 |
| 1537860190 <= VALUES < 1537863790 |
| 1537863790 <= VALUES < 1537867390 |
| 1537867390 <= VALUES < 1537870990 |
| 1537870990 <= VALUES < 1537874590 |
| 1537874590 <= VALUES < 1537878190 |
| VALUES >= 1537878190 |
+-----------------------------------+
Fetched 7 row(s) in 0.02s
##################################################
## drop a range partition
##################################################
ALTER TABLE test_sql_drop DROP RANGE PARTITION VALUES < 1537860190;
##################################################
## between 0-5 minutes after "alter table". Kudu webgui updates the Partition
Schema instantly
##################################################
[:21000] > show range partitions test_sql_drop;
Query: show range partitions test_sql_drop
+-----------------------------------+
| RANGE (flowenddate) |
+-----------------------------------+
| VALUES < 1537860190 |
| 1537860190 <= VALUES < 1537863790 |
| 1537863790 <= VALUES < 1537867390 |
| 1537867390 <= VALUES < 1537870990 |
| 1537870990 <= VALUES < 1537874590 |
| 1537874590 <= VALUES < 1537878190 |
| VALUES >= 1537878190 |
+-----------------------------------+
Fetched 7 row(s) in 0.02s
##################################################
## more than 5 minutes after "alter table".
##################################################
the " show range partitions test_sql_drop;" is updated and correct
smime.p7s
Description: S/MIME Cryptographic Signature
