lidavidm commented on code in PR #37178: URL: https://github.com/apache/arrow/pull/37178#discussion_r1296201718
########## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ########## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.arrow.memory.util; + +import java.util.concurrent.atomic.AtomicInteger; +import java.util.stream.Collectors; + +import org.apache.arrow.memory.BufferAllocator; +import org.apache.arrow.memory.RootAllocator; +import org.apache.arrow.util.VisibleForTesting; + +/** + * Factory class for managing allocators in simple applications that make + * use of an application-wide singleton root allocator with no listeners, + * automatic allocator defaults, and no total allocator limit. + * <p> + * Child allocators are created with a default name with the format + * <code>SingletonAllocator#(AtomicCounter)</code>. + */ +public final class SingletonAllocator { Review Comment: We need to ban this from usage in Arrow itself. I'm also not convinced this is worth shipping as part of Arrow, vs improving documentation on how allocators are supposed to be used. ########## java/memory/memory-core/src/test/java/org/apache/arrow/memory/util/SingletonAllocatorTest.java: ########## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.arrow.memory.util; + +import static org.apache.arrow.memory.util.SingletonAllocator.checkGlobalCleanUpResources; +import static org.apache.arrow.memory.util.SingletonAllocator.getAllocatedMemory; +import static org.apache.arrow.memory.util.SingletonAllocator.getAllocator; +import static org.apache.arrow.memory.util.SingletonAllocator.hasActiveAllocators; +import static org.apache.arrow.memory.util.SingletonAllocator.verify; Review Comment: Don't static import methods. ########## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ########## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.arrow.memory.util; + +import java.util.concurrent.atomic.AtomicInteger; +import java.util.stream.Collectors; + +import org.apache.arrow.memory.BufferAllocator; +import org.apache.arrow.memory.RootAllocator; +import org.apache.arrow.util.VisibleForTesting; + +/** + * Factory class for managing allocators in simple applications that make + * use of an application-wide singleton root allocator with no listeners, + * automatic allocator defaults, and no total allocator limit. Review Comment: I think a better way of describing it is this is a pattern for applications that wish to use a global singleton allocator instead of explicitly passing around the root allocator. It should be preferred to creating a new root allocator on the fly. ########## java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/SingletonAllocator.java: ########## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.arrow.memory.util; + +import java.util.concurrent.atomic.AtomicInteger; +import java.util.stream.Collectors; + +import org.apache.arrow.memory.BufferAllocator; +import org.apache.arrow.memory.RootAllocator; +import org.apache.arrow.util.VisibleForTesting; + +/** + * Factory class for managing allocators in simple applications that make + * use of an application-wide singleton root allocator with no listeners, + * automatic allocator defaults, and no total allocator limit. + * <p> + * Child allocators are created with a default name with the format + * <code>SingletonAllocator#(AtomicCounter)</code>. + */ +public final class SingletonAllocator { + private SingletonAllocator() {} + + private static final RootAllocator ROOT = new RootAllocator(); + private static final AtomicInteger COUNTER = new AtomicInteger(0); + + /** + * Get a new child allocator with a default name, no initial reservation and no maximum allocation. + * + * @return a new child {@link BufferAllocator}. + */ + public static BufferAllocator getAllocator() { + return getAllocator(0, Long.MAX_VALUE); + } + + /** + * Get a new child allocator with a default name and a custom initial reservation and maximum allocation. + * + * @return a new child {@link BufferAllocator}. + */ + public static BufferAllocator getAllocator(final long initReservation, final long maxAllocation) { + return ROOT.newChildAllocator(nextChildName(), initReservation, maxAllocation); + } + + private static String nextChildName() { + return "SingletonAllocator#" + COUNTER.incrementAndGet(); + } + + /** + * Get the total allocated memory across all child allocators. + * + * @return the number of allocated bytes. + */ + public static long getAllocatedMemory() { + return ROOT.getAllocatedMemory(); + } + + /** + * Check if there are any outstanding allocators that have not been closed. + * + * @return <code>true</code> if child allocators exist or <code>false</code> if not. + */ + public static boolean hasActiveAllocators() { + return !ROOT.getChildAllocators().isEmpty(); + } + + @VisibleForTesting + static void checkGlobalCleanUpResources() { Review Comment: Probably this should be public? An application should check that all memory is freed at shutdown ########## docs/source/java/memory.rst: ########## @@ -133,7 +133,7 @@ Development Guidelines Applications should generally: * Use the BufferAllocator interface in APIs instead of RootAllocator. -* Create one RootAllocator at the start of the program. +* Create one RootAllocator at the start of the program (in case `Singleton Allocator`_ is needed, consider using it). Review Comment: I would generally recommend against using it... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
