Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-11-09 Thread Xuneng Zhou
Hi, With a sorted commited.xip array, we could replace the iteration with two binary searches to find the interval to keep. Proposed Optimization - Use binary search to locate the boundaries of XIDs to remove, then compact with a single memmove. The key insight requires under

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-11-06 Thread Xuneng Zhou
Hi, > I conducted several benchmark tests on this patch, and here are some > observations: > > 1) Workloads & settings > # Workload MIXED - Realistic mix of DDL and DML > create_mixed_workload() { > local ROOT="$1" > cat >"$ROOT/mixed_ddl.sql" <<'SQL' > -- DDL workload (catalog changes) > DO $

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-11-06 Thread Xuneng Zhou
Hi, On Fri, Oct 31, 2025 at 6:08 AM Masahiko Sawada wrote: > > On Thu, Oct 30, 2025 at 9:21 AM Xuneng Zhou wrote: > > > > Hi, > > > > On Wed, Oct 29, 2025 at 8:17 PM Xuneng Zhou wrote: > > > > > > Hi, > > > > > > On Sat, Oct 25, 2025 at 2:36 AM Masahiko Sawada > > > wrote: > > > > > > > > On

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-30 Thread Masahiko Sawada
On Thu, Oct 30, 2025 at 9:21 AM Xuneng Zhou wrote: > > Hi, > > On Wed, Oct 29, 2025 at 8:17 PM Xuneng Zhou wrote: > > > > Hi, > > > > On Sat, Oct 25, 2025 at 2:36 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Oct 23, 2025 at 1:17 AM Xuneng Zhou wrote: > > > > > > > > Hi Sawada-san, Michael

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-24 Thread Masahiko Sawada
On Thu, Oct 23, 2025 at 1:17 AM Xuneng Zhou wrote: > > Hi Sawada-san, Michael, > > Thanks for your comments on this patch. > > On Thu, Oct 23, 2025 at 8:28 AM Masahiko Sawada wrote: > > > > On Mon, Oct 20, 2025 at 11:13 PM Xuneng Zhou wrote: > > > > > > Hi, > > > > > > Thanks for looking into th

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-23 Thread Xuneng Zhou
Hi Sawada-san, Michael, Thanks for your comments on this patch. On Thu, Oct 23, 2025 at 8:28 AM Masahiko Sawada wrote: > > On Mon, Oct 20, 2025 at 11:13 PM Xuneng Zhou wrote: > > > > Hi, > > > > Thanks for looking into this. > > > > On Tue, Oct 21, 2025 at 1:05 PM Kirill Reshke > > wrote: > >

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-22 Thread Masahiko Sawada
On Mon, Oct 20, 2025 at 11:13 PM Xuneng Zhou wrote: > > Hi, > > Thanks for looking into this. > > On Tue, Oct 21, 2025 at 1:05 PM Kirill Reshke wrote: > > > > On Tue, 21 Oct 2025 at 04:31, Michael Paquier wrote: > > > > > > On Sat, Oct 18, 2025 at 01:59:40PM +0500, Kirill Reshke wrote: > > > > I

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-20 Thread Xuneng Zhou
Hi, Thanks for looking into this. On Tue, Oct 21, 2025 at 1:05 PM Kirill Reshke wrote: > > On Tue, 21 Oct 2025 at 04:31, Michael Paquier wrote: > > > > On Sat, Oct 18, 2025 at 01:59:40PM +0500, Kirill Reshke wrote: > > > Indeed, these changes look correct. > > > I wonder why b89e151054a0 did th

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-20 Thread Kirill Reshke
On Tue, 21 Oct 2025 at 04:31, Michael Paquier wrote: > > On Sat, Oct 18, 2025 at 01:59:40PM +0500, Kirill Reshke wrote: > > Indeed, these changes look correct. > > I wonder why b89e151054a0 did this place this way, hope we do not miss > > anything here. > > Perhaps a lack of time back in 2014? It

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-20 Thread Michael Paquier
On Sat, Oct 18, 2025 at 01:59:40PM +0500, Kirill Reshke wrote: > Indeed, these changes look correct. > I wonder why b89e151054a0 did this place this way, hope we do not miss > anything here. Perhaps a lack of time back in 2014? It feels like an item where we would need to research a bit some of t

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-20 Thread Xuneng Zhou
Hi, Kirill Reshke 于 2025年10月20日周一 下午6:07写道: > On Mon, 20 Oct 2025 at 13:47, Xuneng Zhou wrote: > > > > Hi, > > > > On Mon, Oct 20, 2025 at 11:36 AM Kirill Reshke > wrote: > > > > > > On Mon, 20 Oct 2025 at 08:08, Xuneng Zhou > wrote: > > > > > > > > Hi, thanks for looking into this. > > > > >

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-20 Thread Kirill Reshke
On Mon, 20 Oct 2025 at 13:47, Xuneng Zhou wrote: > > Hi, > > On Mon, Oct 20, 2025 at 11:36 AM Kirill Reshke wrote: > > > > On Mon, 20 Oct 2025 at 08:08, Xuneng Zhou wrote: > > > > > > Hi, thanks for looking into this. > > > > > > On Sat, Oct 18, 2025 at 4:59 PM Kirill Reshke > > > wrote: > > >

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-20 Thread Xuneng Zhou
Hi, On Mon, Oct 20, 2025 at 11:36 AM Kirill Reshke wrote: > > On Mon, 20 Oct 2025 at 08:08, Xuneng Zhou wrote: > > > > Hi, thanks for looking into this. > > > > On Sat, Oct 18, 2025 at 4:59 PM Kirill Reshke > > wrote: > > > > > > On Sat, 18 Oct 2025 at 12:50, Xuneng Zhou wrote: > > > > > > >

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-19 Thread Kirill Reshke
On Mon, 20 Oct 2025 at 08:08, Xuneng Zhou wrote: > > Hi, thanks for looking into this. > > On Sat, Oct 18, 2025 at 4:59 PM Kirill Reshke wrote: > > > > On Sat, 18 Oct 2025 at 12:50, Xuneng Zhou wrote: > > > > > > Hi Hackers, > > > > Hi! > > > > > The SnapBuildPurgeOlderTxn function previously us

Re: Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-18 Thread Kirill Reshke
On Sat, 18 Oct 2025 at 12:50, Xuneng Zhou wrote: > > Hi Hackers, Hi! > The SnapBuildPurgeOlderTxn function previously used a suboptimal > method to remove old XIDs from the committed.xip array. It allocated a > temporary workspace array, copied the surviving elements into it, and > then copied t

Optimize SnapBuildPurgeOlderTxn: use in-place compaction instead of temporary array

2025-10-18 Thread Xuneng Zhou
Hi Hackers, The SnapBuildPurgeOlderTxn function previously used a suboptimal method to remove old XIDs from the committed.xip array. It allocated a temporary workspace array, copied the surviving elements into it, and then copied them back, incurring unnecessary memory allocation and multiple data