RussellSpitzer commented on a change in pull request #4126:
URL: https://github.com/apache/iceberg/pull/4126#discussion_r806987049
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java
##########
@@ -151,4 +161,25 @@ TableCatalog tableCatalog() {
return tableCatalog;
}
}
+
+ /**
+ * Closes the procedure's executor service if a new one was created.
+ */
+ protected void closeService() {
+ if (executorService != null) {
+ executorService.shutdown();
+ }
+ }
+
+ protected ExecutorService executorService(int concurrentDeletes, String
nameFormat) {
+ Preconditions.checkArgument(executorService == null, "Cannot create a new
executor service, one already exists.");
+ executorService = MoreExecutors.getExitingExecutorService(
+ (ThreadPoolExecutor) Executors.newFixedThreadPool(
+ concurrentDeletes,
+ new ThreadFactoryBuilder()
+ .setNameFormat(nameFormat + "-%d")
Review comment:
Sure I can protect against that, this is an internal method so i'm not
that worried.
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java
##########
@@ -151,4 +161,25 @@ TableCatalog tableCatalog() {
return tableCatalog;
}
}
+
+ /**
+ * Closes the procedure's executor service if a new one was created.
+ */
+ protected void closeService() {
+ if (executorService != null) {
+ executorService.shutdown();
Review comment:
Sure we could set it to null, again these are pretty private methods so
I'm not to worried about usages.
I'm not sure what you are asking for with the synchronize here, do you want
it to be
shutdown?
awaitTermination?
I think in this usage that the pool had better be out of work by the time we
get to this point since we have already returned results but we can do that
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java
##########
@@ -151,4 +161,25 @@ TableCatalog tableCatalog() {
return tableCatalog;
}
}
+
+ /**
+ * Closes the procedure's executor service if a new one was created.
+ */
+ protected void closeService() {
+ if (executorService != null) {
+ executorService.shutdown();
Review comment:
I can add a
Preconditions.checkArgument(executorService.isTerminated(),
"Procedure error, close service called before service was
finished");
Which will cause this to break if it is used incorrectly by
iceberg devs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
