morningman commented on a change in pull request #1792: Add ChunkAllocator to accelerate chunk allocation URL: https://github.com/apache/incubator-doris/pull/1792#discussion_r323773369
########## File path: be/src/runtime/memory/chunk_allocator.h ########## @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#pragma once + +#include <atomic> +#include <memory> +#include <vector> +#include <cstddef> +#include <cstdint> + +namespace doris { + +class Chunk; +class ChunkArena; + +// Used to allocate memory with power-of-two length. +// This Allocator allocate memory from system and cache free chunks for +// later use. +// +// ChunkAllocator has one ChunkArena for each CPU core, it will try to allocate +// memory from current core arena firstly. In this way, there will be no lock contention +// between concurrently-running threads. If this fails, ChunkAllocator will try to allocate +// memroy from other core's arena. +// +// Memory Reservation +// ChunkAllocator has a limit about how much free chunk bytes it can reserve, above which +// chunk will released to system memory. For the worst case, when the limits is 0, it will +// act as allocating directly from system. +// +// ChunkArena will keep a separate free list for each chunk size. In common case, chunk will +// be allocated from current core arena. In this case, there is no lock contention. +// +// Must call CpuInfo::init() and DorisMetrics::initialize() to achieve good performance +// before first object is created. And call init_instance() before use instance is called. +class ChunkAllocator { +public: + static void init_instance(size_t reserve_limit); + +#if BE_TEST + static ChunkAllocator* instance(); +#else + static ChunkAllocator* instance() { + return _s_instance; + } +#endif + + ChunkAllocator(size_t reserve_limit); + + // Allocate a Chunk with a power-of-two length "size". + // Rreturn true if success and allocated chunk is saved in "chunk". Review comment: ```suggestion // Return true if success and allocated chunk is saved in "chunk". ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
