kbendick commented on a change in pull request #4126:
URL: https://github.com/apache/iceberg/pull/4126#discussion_r806362061
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java
##########
@@ -151,4 +161,25 @@ TableCatalog tableCatalog() {
return tableCatalog;
}
}
+
+ /**
+ * Closes the procedure's executor service if a new one was created.
+ */
+ protected void closeService() {
+ if (executorService != null) {
+ executorService.shutdown();
Review comment:
Should you set it to null here after shutting down? And potentially
synchronize?
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java
##########
@@ -151,4 +161,25 @@ TableCatalog tableCatalog() {
return tableCatalog;
}
}
+
+ /**
+ * Closes the procedure's executor service if a new one was created.
+ */
+ protected void closeService() {
+ if (executorService != null) {
+ executorService.shutdown();
Review comment:
The mention of `synchronize` can be ignored. I was just covering all my
bases and was mostly concerned about `closeService` getting called multiple
times. But I can see how that would be unlikely to happen.
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java
##########
@@ -151,4 +161,25 @@ TableCatalog tableCatalog() {
return tableCatalog;
}
}
+
+ /**
+ * Closes the procedure's executor service if a new one was created.
+ */
+ protected void closeService() {
+ if (executorService != null) {
+ executorService.shutdown();
+ }
+ }
+
+ protected ExecutorService executorService(int concurrentDeletes, String
nameFormat) {
+ Preconditions.checkArgument(executorService == null, "Cannot create a new
executor service, one already exists.");
+ executorService = MoreExecutors.getExitingExecutorService(
+ (ThreadPoolExecutor) Executors.newFixedThreadPool(
+ concurrentDeletes,
+ new ThreadFactoryBuilder()
+ .setNameFormat(nameFormat + "-%d")
Review comment:
Agreed that it's not too much of a worry. Based on the usage, this is
all very much internal. So somebody would have to explicitly pass in `null` as
the argument. But a `Preconditions` check isn't a bad idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
