Hi Weijian, First of all, great start! I and probably someone else will look at the PR shared. Thanks for sharing it. A good place to look at next would be how memory allocation and deallocation works as part of a memory-intensive operator such as Hybrid Hash Join or Sort (use in some operators such as group by). There are a couple of resources that may be helpful: - https://scholarcommons.scu.edu/cgi/viewcontent.cgi?params=/context/cseng_mstr/article/1034/&path_info=Giulliano_Silva_Zanotti_Siviero_Thesis.pdf
- Diane L. Davison and Goetz Graefe. “Dynamic Resource Brokering for Multi-User Query Execution”. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data. SIGMOD ’95. San Jose, California, USA: Association for Computing Machinery, 1995, pp. 281–292. isbn: 0897917316. doi: 10.1145/223784.223845. url: https://doi.org/10.1145/223784.223845. Please feel free to share your summarization or your proposal so I can have a look. I heard somewhere the deadline for submitting it is March 28th? Not sure but double check not to miss it. Best, Shiva On Tue, Mar 24, 2026 at 8:47 AM viking deng <[email protected]> wrote: > Hi Shiva, > > I’m preparing my application for the Apache AsterixDB GSoC 2026 project on > Dynamic Memory Management, and I wanted to share one concrete contribution > I completed while studying the codebase. > > I’m currently a Master’s student in Computer Science at Beihang University. > I previously interned at Shopee and ByteDance, where I worked on backend > systems and infrastructure-related development. I’m particularly interested > in systems, runtime behavior, and resource management, which is why this > AsterixDB project strongly appeals to me. > > While studying the AsterixDB/Hyracks runtime, I identified a real issue in > the NC-side memory path and opened the following PR: > > https://github.com/apache/asterixdb/pull/42 > > This patch fixes a runtime-side bug in Hyracks NC memory accounting, > specifically in org.apache.hyracks.control.nc.Joblet. > > Joblet tracks outstanding frame memory in memoryAllocation, and close() > uses that value to detect and reclaim leaked memory. In the previous code, > however, deallocateFrames(int) increased memoryAllocation instead of > decreasing it. As a result, even a balanced allocate/deallocate sequence > could still leave a positive outstanding-memory count, and close() could > incorrectly treat that memory as leaked and deallocate it again. > > I also fixed a related issue in the same path: if > MemoryManager.allocate(bytes) succeeded but > FrameManager.allocateFrame(bytes) later failed, the reservation was not > rolled back. That could leave a phantom allocation behind and again > interfere with leak cleanup. > > I added focused unit tests for both cases: > > - a balanced alloc/free sequence should not trigger an extra > deallocation on close() > - a failed frame allocation should immediately release its reservation > > I verified the patch with module test compilation and by directly running > the new JobletTest, and the tests passed. > > I chose this issue because it is part of the execution/runtime memory path > rather than a setup issue, and working through it helped me better > understand the NC-side memory lifecycle in Hyracks. That code-reading > process has also been directly useful for forming my understanding of how > dynamic operator memory management could be introduced more systematically. > > I am continuing to study the current budgeting and memory-intensive > operator path in AsterixDB/Hyracks, especially around runtime memory > ownership, operator-level memory behavior, and where a future > broker/resizing loop could be integrated safely. > > If helpful, I would also be glad to summarize my current understanding of > the next implementation steps for the project. > > Best regards, > Weijian Deng > -- Shiva Jahangiri Assistant Professor in Computer Science and Engineering Department Santa Clara University
