>From Ritik Raj <ritik....@couchbase.com>: Attention is currently required from: Murtadha Hubail. Ritik Raj has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249 )
Change subject: [ASTERIXDB-3636][STO] Introducing column buffer pool ...................................................................... Patch Set 6: (23 comments) Commit Message: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/46da67fc_d78ff216 PS4, Line 9: - user model changes: no : - storage format changes: no : - interface changes: no > update and add details Done File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/696b58b1_85c12ff8 PS4, Line 333: , hence choosing > ; using Done File asterixdb/asterix-column/src/main/java/org/apache/asterix/column/operation/lsm/flush/FlushColumnMetadata.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/95339ccf_38a04473 PS4, Line 460: //Dereference multiPageOp > move this down Done File asterixdb/asterix-column/src/main/java/org/apache/asterix/column/operation/lsm/merge/MergeColumnTupleWriter.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/05455a88_f67ce584 PS4, Line 248: onHouse > remove onHouse Done File asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/StorageProperties.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/e8f96e65_87a00b49 PS4, Line 75: STORAGE_COLUMN_BUFFER_GRANULE_SIZE > STORAGE_COLUMN_BUFFER_SIZE Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/33d5812e_d2e2ede8 PS4, Line 76: STORAGE_COLUMN_BUFFER_POOL_CAPPED_SIZE_PERCENTAGE > STORAGE_COLUMN_BUFFERS_MAX_SIZE_PERCENTAGE Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/2c52c9de_44835fb5 PS4, Line 77: STORAGE_COLUMN_BUFFER_POOL_SIZE > STORAGE_COLUMN_BUFFER_POOL_MAX_SIZE Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/b15c0191_9cbbfa40 PS4, Line 169: case STORAGE_COLUMN_BUFFER_GRANULE_SIZE: : return "The size of the granule for the column buffer pool. This is the size of each buffer in the pool."; : case STORAGE_COLUMN_BUFFER_POOL_CAPPED_SIZE_PERCENTAGE: : return "The percentage of the total memory allocated to the column buffer pool. This is used to " : + "control the size of the column buffer pool relative to the total runtime available memory."; : case STORAGE_COLUMN_BUFFER_POOL_SIZE: : return "The count of buffers in the column buffer pool."; > update accordingly Done File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/api/IColumnWriteMultiPageOp.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/f41b7a2f_e3ab566b PS4, Line 61: confiscateTemporaryOnHouseBuffer > remove OnHouse Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/7691e814_59771bfc PS4, Line 67: releaseOnHouseBuffer > remove OnHouse Done File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/ColumnBufferPool.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/64df81ad_c70f93ab PS4, Line 72: // Stats dumper thread : statsDumpScheduler = Executors.newSingleThreadScheduledExecutor(r -> { : Thread t = new Thread(r, "ColumnBufferPoolStatsDumper"); : t.setDaemon(true); : return t; : }); : startStatsDumpThread(); > If you need those stats, make this a TimerTask Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/f05c9ad0_0ee331eb PS4, Line 155: if (totalAllocatedMemoryInBytes.get() + (long) poolSize * columnBufferGranuleInBytes > cappedMemoryInBytes) { > ensure.... Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/e0e8e7cf_e65b922a PS4, Line 156: throw new OutOfMemoryError("Cannot allocate more buffers, memory limit reached."); > log error with all details (stats). […] Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/dcf0ec2f_d9d8f5bf PS4, Line 160: // grab a new buffer > Remember, a comment should tell you 'why', not 'what' Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/1692facf_1f8bc02d PS4, Line 171: if (buffer == null) { : // unreachable : return; : } > if you are 100% sure, this isn't reachable, it should be illegal state > exception to catch issues Done File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/IColumnBufferPool.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/add8e976_579f10b8 PS4, Line 23: IColumnBufferPool > Try to simplify this Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/64eeb06d_5343063c PS4, Line 34: reserveCredits > reserve Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/9c64999d_3ffb55c1 PS4, Line 41: releaseCredits > release Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/ead38f7e_b6f97834 PS4, Line 49: reserveExtraCredits > reserveExtraBuffers Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/6d605e86_f020051b PS4, Line 58: tryAcquiringExtraCredits > tryReserveExtraBuffers Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/661f275e_389ae19b PS4, Line 66: acquireBuffer > get Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/e09b9f66_b0442b82 PS4, Line 72: releaseBuffer > recycle Done File hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-common-test/src/test/java/org/apache/hyracks/storage/common/ColumnBufferPoolTest.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249/comment/cf96dcdf_8314f2ea PS4, Line 21: .* > replace this Done -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20249 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: ionic Gerrit-Change-Id: I62437839d89b11d950e6715a00e844aedd0dab8e Gerrit-Change-Number: 20249 Gerrit-PatchSet: 6 Gerrit-Owner: Ritik Raj <ritik....@couchbase.com> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <jenk...@fulliautomatix.ics.uci.edu> Gerrit-CC: Murtadha Hubail <mhub...@apache.org> Gerrit-Attention: Murtadha Hubail <mhub...@apache.org> Gerrit-Comment-Date: Tue, 26 Aug 2025 08:16:22 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: Murtadha Hubail <mhub...@apache.org> Gerrit-MessageType: comment
