Hi Shiva, Thank you very much for the guidance.
That is very helpful. I will study the memory allocation and deallocation path in memory-intensive operators next, especially Hybrid Hash Join, Sort, and the related Group By path, and use that to refine my understanding of where dynamic memory management could be integrated safely. The references are also very helpful — I’ll read them carefully. I’m currently refining both a short summary and my proposal, and I’ll share them with you soon for feedback. By the way, the deadline maybe 3.31. Best regards, Weijian Deng Shiva Jahangiri <[email protected]> 于2026年3月25日周三 12:47写道： > Hi Weijian, > > First of all, great start! I and probably someone else will look at the PR > shared. Thanks for sharing it. > A good place to look at next would be how memory allocation and > deallocation works as part of a memory-intensive operator such as > Hybrid Hash Join or Sort (use in some operators such as group by). > There are a couple of resources that may be helpful: > - > > https://scholarcommons.scu.edu/cgi/viewcontent.cgi?params=/context/cseng_mstr/article/1034/&path_info=Giulliano_Silva_Zanotti_Siviero_Thesis.pdf > > - Diane L. Davison and Goetz Graefe. “Dynamic Resource Brokering for > Multi-User Query Execution”. In: > > Proceedings of the 1995 ACM SIGMOD International Conference on Management > of Data. SIGMOD ’95. > > San Jose, California, USA: Association for Computing Machinery, 1995, pp. > 281–292. isbn: 0897917316. doi: > > 10.1145/223784.223845. url: https://doi.org/10.1145/223784.223845. > > > Please feel free to share your summarization or your proposal so I can have > a look. I heard somewhere the deadline for submitting it is March 28th? Not > sure but double check not to miss it. > > > Best, > > Shiva > > > > On Tue, Mar 24, 2026 at 8:47 AM viking deng <[email protected]> wrote: > > > Hi Shiva, > > > > I’m preparing my application for the Apache AsterixDB GSoC 2026 project > on > > Dynamic Memory Management, and I wanted to share one concrete > contribution > > I completed while studying the codebase. > > > > I’m currently a Master’s student in Computer Science at Beihang > University. > > I previously interned at Shopee and ByteDance, where I worked on backend > > systems and infrastructure-related development. I’m particularly > interested > > in systems, runtime behavior, and resource management, which is why this > > AsterixDB project strongly appeals to me. > > > > While studying the AsterixDB/Hyracks runtime, I identified a real issue > in > > the NC-side memory path and opened the following PR: > > > > https://github.com/apache/asterixdb/pull/42 > > > > This patch fixes a runtime-side bug in Hyracks NC memory accounting, > > specifically in org.apache.hyracks.control.nc.Joblet. > > > > Joblet tracks outstanding frame memory in memoryAllocation, and close() > > uses that value to detect and reclaim leaked memory. In the previous > code, > > however, deallocateFrames(int) increased memoryAllocation instead of > > decreasing it. As a result, even a balanced allocate/deallocate sequence > > could still leave a positive outstanding-memory count, and close() could > > incorrectly treat that memory as leaked and deallocate it again. > > > > I also fixed a related issue in the same path: if > > MemoryManager.allocate(bytes) succeeded but > > FrameManager.allocateFrame(bytes) later failed, the reservation was not > > rolled back. That could leave a phantom allocation behind and again > > interfere with leak cleanup. > > > > I added focused unit tests for both cases: > > > > - a balanced alloc/free sequence should not trigger an extra > > deallocation on close() > > - a failed frame allocation should immediately release its reservation > > > > I verified the patch with module test compilation and by directly running > > the new JobletTest, and the tests passed. > > > > I chose this issue because it is part of the execution/runtime memory > path > > rather than a setup issue, and working through it helped me better > > understand the NC-side memory lifecycle in Hyracks. That code-reading > > process has also been directly useful for forming my understanding of how > > dynamic operator memory management could be introduced more > systematically. > > > > I am continuing to study the current budgeting and memory-intensive > > operator path in AsterixDB/Hyracks, especially around runtime memory > > ownership, operator-level memory behavior, and where a future > > broker/resizing loop could be integrated safely. > > > > If helpful, I would also be glad to summarize my current understanding of > > the next implementation steps for the project. > > > > Best regards, > > Weijian Deng > > > > > -- > Shiva Jahangiri > Assistant Professor in Computer Science and Engineering Department > Santa Clara University >
