Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-29 Thread Mengxing Liu
in it, and I cannot estimate how much time it would take. > -Original Messages- > From: "Kevin Grittner" <kgri...@gmail.com> > Sent Time: 2017-03-28 00:16:11 (Tuesday) > To: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.cn> > Cc: "pgsql-h

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-25 Thread Mengxing Liu
tner" <kgri...@gmail.com> > Sent Time: 2017-03-25 04:53:36 (Saturday) > To: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.cn> > Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org> > Subject: Re: [HACKERS] Guidelines for GSoC student

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-15 Thread Mengxing Liu
> -Original Messages- > From: "Kevin Grittner" <kgri...@gmail.com> > Sent Time: 2017-03-15 23:20:07 (Wednesday) > To: DEV_OPS <dev...@ww-it.cn> > Cc: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.cn>, > "pgsql-hackers@postgresq

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-22 Thread Mengxing Liu
sal, not ready-to-go proposal documents. > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Mengxing Liu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-30 Thread Mengxing Liu
ared memory issues, since the structure *must* > live in shared memory. > Thank you. If there is no other problem, I will submit the proposal. -- Mengxing Liu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-14 Thread Mengxing Liu
ement. For me, the challenge is profiling the execution. Is there any tools in PostgreSQL to analysis where is the CPU cycles consumed? Or I have to instrument and time by myself? Regards. -- Mengxing Liu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-11 Thread Mengxing Liu
> -Original Messages- > From: "Kevin Grittner" <kgri...@gmail.com> > Sent Time: 2017-03-12 04:24:29 (Sunday) > To: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.cn> > Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql

Re: [HACKERS] GSOC Introduction / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-10 Thread Mengxing Liu
Hi George, I am Mengxing Liu. Happy to meet someone with the same idea : ) I have been concentrating on it for a long time, reading papers, reading source codes, and discussing details with Mr Grittner. So I really understand your passion on it. But definitely I don't want all these effects

[HACKERS] [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-10 Thread Mengxing Liu
Hi, all My name is Mengxing Liu. I am interested in the project "Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions”. After discussing with Kevin off-list, I think it's time to post discussion here. I am afraid that there are two things that I need your

[HACKERS] [GOSC' 17][Performance report] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-07-31 Thread Mengxing Liu
nce. It's hard to explain why... You can reply directly if you have any questions or comments. -- Sincerely Mengxing Liu HTAB-for-conflict-tracking.patch Description: Binary data skip-list-for-conflict-tracking.patch Description: Binary data reduce-content

[HACKERS] Re: [GSOC][weekly report 9] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-08-14 Thread Mengxing Liu
me implementations. What's your opinion? > -- > Álvaro Herrerahttps://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sincerely Mengxing Liu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] [GSOC][weekly report 6] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-07-16 Thread Mengxing Liu
list is enough now. I'll make a clean patch after I figure out bugs. -- Sincerely Mengxing Liu

[HACKERS] [GSOC][weekly report 5] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-07-09 Thread Mengxing Liu
. To focus on this issue, I removed other parts of modification. Do you have any advices for this ? -- Sincerely Mengxing Liu finishedlock.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

[HACKERS] [GSOC][weekly report 7] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-07-23 Thread Mengxing Liu
for conflict tracking. 2) reducing the global lock contention 3) using skip list for conflict tracking. But all of them did not improve the performance. So I'm a little confused now about what to do next. Could you please give me any suggestions? -- Sincerely Mengxing Liu skip-list

[HACKERS] [GSOC] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-07-26 Thread Mengxing Liu
means, the skip list can work! Looking forward to your advices. -- Sincerely Mengxing Liu skip-list-for-conflict-tracking.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] [GSOC][weekly report 4] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-28 Thread Mengxing Liu
. But I'm really looking forward to your advice for my proposal. We can't be too careful to modify the code related to locks. -- Sincerely Mengxing Liu

Re: [HACKERS] [GSOC] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-07-30 Thread Mengxing Liu
ed by some mistakes. When I changed a machine to run the code, this phenomenon didn't happen anymore.. -Original Messages- From: "Robert Haas" <robertmh...@gmail.com> Sent Time: 2017-07-29 02:46:47 (Saturday) To: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.c

[HACKERS] [GSOC][weekly report 8] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-07-30 Thread Mengxing Liu
RWConflictExists/SetRWConflict is as below. | | RWConflictExists | SetRWConflict | | linked list | 1.39% | 0.14% | | skip list | 1.15% | 0.35% | According to the suggestions of Alvaro, I'll give a detailed performance report tomorrow. -- Sincerely Mengxing Liu skip-list-for-conflict

[HACKERS] Re: [GSOC][weekly report 9] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-08-08 Thread Mengxing Liu
> -- > Álvaro Herrerahttps://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sincerely Mengxing Liu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] [GSOC][weekly report 9] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-08-07 Thread Mengxing Liu
and the performance can't be improved. A chart is attached. This optimization can achieve 14% speedup at most. Do you think this optimization can be accepted? -- Sincerely Mengxing Liu skip-list-for-conflict-tracking.patch Description: Binary data -- Sent via pgsql-hackers mailing

[HACKERS] [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-05-09 Thread Mengxing Liu
s once a week? > What steps will be taken next during the community bonding period. As I wrote in the proposal, I want to establish the environment and read the related source code. Do you have any suggestions for me? -- Mengxing Liu

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-07 Thread Mengxing Liu
ms to go to lightweight locking. Since > you said this is a CPU graph, that again suggests spinlock contention > issues. > > -- Yes. Is there any other kind of locks besides spinlock? I'm reading locks in PG now. If all locks are spinlock, the CPU should be used 100%. But now only 50%

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-08 Thread Mengxing Liu
think that's why PG is so slow in high contention environment. > -Original Messages- > From: "Robert Haas" <robertmh...@gmail.com> > Sent Time: 2017-06-08 01:30:58 (Thursday) > To: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.cn> > Cc: kgrit

Re: [HACKERS] [GSOC][weekly report 3] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-21 Thread Mengxing Liu
ause hash table objects also consume memory ) But just for performance, it would be less efficient than my patch. Because it has to maintain linked lists, besides hash tables. In other words, it does more works than my patch. Another point is that removing linked list may have more opportunit

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-02 Thread Mengxing Liu
ny chance to get rid of this problem? BTW, I found that the email is not very convenient, especially when I have some problem and want to discuss with you. Would you mind scheduling a meeting every week by Skye, or other instant message software you like? I would not take you too much time. Ma

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-03 Thread Mengxing Liu
> -Original Messages- > From: "Kevin Grittner" <kgri...@gmail.com> > Sent Time: 2017-06-03 01:44:16 (Saturday) > To: "Alvaro Herrera" <alvhe...@2ndquadrant.com> > Cc: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.cn>,

[HACKERS] [GSOC] [Weekly report 2] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-14 Thread Mengxing Liu
, PreCommit_CheckForSerializationFailure Sincerely -- Mengxing Liu

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-06 Thread Mengxing Liu
ot; to profile is really ugly. Perf lock can only record kernel mutex, and requires kernel support, so I didn't use it. Do you have any good idea about profiling time waiting for locks? > -Original Messages- > From: "Mengxing Liu" <liu-m...@mails.tsinghua.edu.cn> > S