lgbo-ustc commented on issue #8003: URL: https://github.com/apache/incubator-gluten/issues/8003#issuecomment-2487267471
We track the memory usage of hash table in the join ``` 2024-11-20 10:58:49.163 <Error> GraceHashJoin: xxx total_rows: 586379, total_bytes: 792206888 2024-11-20 10:58:49.171 <Error> GraceHashJoin: xxx total_rows: 590411, total_bytes: 796729200 2024-11-20 10:58:49.180 <Error> GraceHashJoin: xxx total_rows: 594443, total_bytes: 801251512 2024-11-20 10:58:49.189 <Error> GraceHashJoin: xxx total_rows: 598475, total_bytes: 805773824 2024-11-20 10:58:49.197 <Error> GraceHashJoin: xxx total_rows: 602507, total_bytes: 810296136 Exception 2024-11-20 10:58:49.200 <Error> local_engine: Enter java exception handle. in thread "Executor task launch worker for task 0.0 in stage 11.0 (TID 1040)" org.apache.gluten.exception.GlutenException: Memory limit exceeded: would use 1.50 GiB (attempt to allocate chunk of 4440521 bytes), current RSS 2.80 GiB, maximum: 1.50 GiB. 0. ../contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000001469db99 1. ./build/../src/Common/Exception.cpp:109: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000069da63c 2. ../src/Common/Exception.h:111: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x00000000068ca54c 3. ../src/Common/Exception.h:129: DB::Exception::Exception<char const*, char const*, String, long&, String, String, char const*, std::basic_string_view<char, std::char_traits<char>>>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<long&>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<char const*>::type, std::type_identity<std::basic_string_view<char, std::char_traits<char>>>::type>, char const*&&, char const*&&, String&&, long&, String&&, String&&, char const*&&, std::basic_string_view<char, std::char_traits<char>>&&) @ 0x00000000069ea0c9 4. ./build/../src/Common/MemoryTracker.cpp:326: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x00000000069e8ee1 5. ./build/../src/Common/MemoryTracker.cpp:383: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x00000000069e8a96 6. ./build/../src/Common/CurrentMemoryTracker.cpp:64: CurrentMemoryTracker::alloc(long) @ 0x00000000069ccb1f 7. ./build/../src/Common/Allocator.cpp:233: Allocator<false, false>::realloc(void*, unsigned long, unsigned long, unsigned long) @ 0x00000000069bab7e 8. ../src/Common/PODArray.h:152: void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 63ul, 64ul>::resize<>(unsigned long) @ 0x0000000006a44e40 9. ./build/../src/Columns/ColumnString.cpp:156: DB::ColumnString::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long) @ 0x00000000102f5b49 10. ./build/../src/Columns/ColumnTuple.cpp:370: DB::ColumnTuple::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long) @ 0x000000001031e7e0 11. ./build/../src/Columns/ColumnArray.cpp:605: DB::ColumnArray::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long) @ 0x0000000010195bf7 12. ./build/../utils/extern-local-engine/Storages/IO/NativeReader.cpp:150: local_engine::readNormalComplexData(DB::ReadBuffer&, COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, local_engine::NativeReader::ColumnParseUtil&) @ 0x0000000006e5e0b5 13. ../contrib/llvm-project/libcxx/include/__functional/function.h:848: ? @ 0x0000000006e5d8d0 14. ./build/../utils/extern-local-engine/Storages/IO/NativeReader.cpp:71: local_engine::NativeReader::read() @ 0x0000000006e5be69 15. ./build/../utils/extern-local-engine/Shuffle/ShuffleReader.cpp:51: local_engine::ShuffleReader::read() @ 0x0000000006f47122 16. ./build/../utils/extern-local-engine/local_engine_jni.cpp:554: Java_org_apache_gluten_vectorized_CHStreamReader_nativeNext @ 0x00000000068b61d7 ``` Total memory limit is 1.5G, but the memory used by the hash table has been 800M. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
