Hi Quanlong We don't have any loadbalancing proxy to the impalads. And we mainly use JDBC or the impala-shell to connect to just one impalad (coordinator).
As we have CDH5.15 we have changed now to 38 executors (no coordinator) and just one additional coordinator (without executor). The statestore logs no messages at all (except the boot and subscriber connect messages). So, to make a long story short. The problem still exists, at least after table creation and the first ALTER TABLE modification. However, it seems that after that it works without any delay. We created two test-tables without any data, both showed the same behavior. Cheers Josef On 07.10.18, 09:03, "Quanlong Huang" <[email protected]> wrote: Hi Josef, Does the client connect to impalads via a load-balancing proxy? If so, you might need to set SYNC_DDL=1 before triggering any DDLs. BTW, 5 minutes seems too long. Do you see any warnings or errors in statestored's log? Which Impala version do you use? After Impala-2.9 (also distributed in CDH5.12), you can set up coordinator/executor roles for impalads ( https://www.cloudera.com/documentation/enterprise/5-12-x/topics/impala_dedicated_coordinator.html). Leave a few impalads to be coordinator and the others to be executor may help. Quanlong On Mon, Oct 1, 2018 at 10:46 PM <[email protected]> wrote: > Hi guys > > > > I’m trying to do an ALTER TABLE for a Kudu range partitioned table. The > ALTER TABLE itself works fine, my issue is that most of the time I get a > delay of about 5min until the “show range partitions xyz” shows the > current/updated range partitions and we need this show command to > automatically update the table ranges and cut the range boundaries... > > > > Sometimes the delay is there and sometimes not - at least after a restart > of impala. I’ve tried several things, removed the kudu table, dropped the > table in impala and re-added it plus other things, but I don’t get a > reproducible result. sometimes it’s fine, but most of the time it’s not. > > > > If the issue persists, the two following commands doesn’t help to get the > table updated: `REFRESH` and `INVALIDATE METADAT` > > > > We have 33 Impala nodes. > > > > Any other ideas to solve the issue? Or is it well known. Thanks in advance > > > > Josef > > > > > > > > *Example:* > > ################################################## > > ## Kudu Partition Schema: > > ################################################## > > > > HASH (flowEndDate, uniqueID) PARTITIONS 16, > > RANGE (flowEndDate) ( > > PARTITION VALUES < 1537860190, > > PARTITION 1537860190 <= VALUES < 1537863790, > > PARTITION 1537863790 <= VALUES < 1537867390, > > PARTITION 1537867390 <= VALUES < 1537870990, > > PARTITION 1537870990 <= VALUES < 1537874590, > > PARTITION 1537874590 <= VALUES < 1537878190, > > PARTITION VALUES >= 1537878190 > > ) > > > > ################################################## > > ## show partitions > > ################################################## > > > > [:21000] > show range partitions test_sql_drop; > > Query: show range partitions test_sql_drop > > +-----------------------------------+ > > | RANGE (flowenddate) | > > +-----------------------------------+ > > | VALUES < 1537860190 | > > | 1537860190 <= VALUES < 1537863790 | > > | 1537863790 <= VALUES < 1537867390 | > > | 1537867390 <= VALUES < 1537870990 | > > | 1537870990 <= VALUES < 1537874590 | > > | 1537874590 <= VALUES < 1537878190 | > > | VALUES >= 1537878190 | > > +-----------------------------------+ > > Fetched 7 row(s) in 0.02s > > > > > > ################################################## > > ## drop a range partition > > ################################################## > > > > ALTER TABLE test_sql_drop DROP RANGE PARTITION VALUES < 1537860190; > > > > > > ################################################## > > ## between 0-5 minutes after "alter table". Kudu webgui updates the > Partition Schema instantly > > ################################################## > > > > [:21000] > show range partitions test_sql_drop; > > Query: show range partitions test_sql_drop > > +-----------------------------------+ > > | RANGE (flowenddate) | > > +-----------------------------------+ > > | VALUES < 1537860190 | > > | 1537860190 <= VALUES < 1537863790 | > > | 1537863790 <= VALUES < 1537867390 | > > | 1537867390 <= VALUES < 1537870990 | > > | 1537870990 <= VALUES < 1537874590 | > > | 1537874590 <= VALUES < 1537878190 | > > | VALUES >= 1537878190 | > > +-----------------------------------+ > > Fetched 7 row(s) in 0.02s > > > > ################################################## > > ## more than 5 minutes after "alter table". > > ################################################## > > > > the " show range partitions test_sql_drop;" is updated and correct >
smime.p7s
Description: S/MIME Cryptographic Signature
