clintropolis commented on a change in pull request #9523: Ability to Delete
task logs and segments from Azure Storage
URL: https://github.com/apache/druid/pull/9523#discussion_r394183307
##########
File path:
extensions-core/azure-extensions/src/main/java/org/apache/druid/storage/azure/AzureTaskLogs.java
##########
@@ -151,14 +167,36 @@ private String getTaskReportsKey(String taskid)
}
@Override
- public void killAll()
+ public void killAll() throws IOException
{
- throw new UnsupportedOperationException("not implemented");
+ log.info("Deleting all task logs from Azure storage location [bucket: %s
prefix: %s].",
+ config.getContainer(), config.getPrefix()
+ );
+
+ long now = timeSupplier.getAsLong();
+ killOlderThan(now);
}
@Override
- public void killOlderThan(long timestamp)
+ public void killOlderThan(long timestamp) throws IOException
{
- throw new UnsupportedOperationException("not implemented");
+ log.info("Deleting all task logs from Azure storage location [bucket: '%s'
prefix: '%s'] older than %s.",
+ config.getContainer(), config.getPrefix(), new Date(timestamp)
+ );
+ try {
+ AzureUtils.deleteObjectsInPath(
+ azureStorage,
+ inputDataConfig,
+ accountConfig,
+ azureCloudBlobIterableFactory,
+ config.getContainer(),
+ config.getPrefix(),
+ (object) -> object.getLastModifed().getTime() < timestamp
+ );
+ }
+ catch (Exception e) {
+ log.error("Error occurred while deleting task log files from s3. Error:
%s", e.getMessage());
Review comment:
same wrong cloud
```suggestion
log.error("Error occurred while deleting task log files from Azure.
Error: %s", e.getMessage());
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
