KKcorps opened a new issue, #10603:
URL: https://github.com/apache/pinot/issues/10603
The table deletion flow was changed in #8422 where we send a message to
servers and let them take care of waiting for EV convergence and then remove
the segments
On the controller side, we only takes care of deleting the configs and ideal
states.
However, this leads to a lot of exceptions.
For eg. when servers try to close consuming segments it starts getting
`TableNotFound` exception
The new deletion flow already has the following TODO which will help solve
this issue.
```java
public void deleteOfflineTable(String tableName, @Nullable String
retentionPeriod) {
String offlineTableName =
TableNameBuilder.OFFLINE.tableNameWithType(tableName);
LOGGER.info("Deleting table {}: Start", offlineTableName);
// Remove the table from brokerResource
HelixHelper.removeResourceFromBrokerIdealState(_helixZkManager,
offlineTableName);
LOGGER.info("Deleting table {}: Removed from broker resource",
offlineTableName);
// Drop the table on servers
// TODO: Make this api idempotent and blocking by waiting for
externalview to converge on controllers
// instead of servers. This is because if externalview gets updated
with significant delay,
// we may have the race condition for table recreation that the new
table will use the old states
// (old table data manager) on the servers.
// Steps needed:
// 1. Drop the helix resource first (set idealstate as null)
// 2. Wait for the externalview to converge
// 3. Get servers for the tenant, and send delete table message to
these servers
deleteTableOnServer(offlineTableName);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
