On 6/5/23 11:33 AM, Heikki Linnakangas wrote:
On 05/06/2023 11:18, Tom Lane wrote:Heikki Linnakangas <hlinn...@iki.fi> writes:I spoke with some folks at PGCon about making PostgreSQL multi-threaded, so that the whole server runs in a single process, with multiple threads. It has been discussed many times in the past, last thread onpgsql-hackers was back in 2017 when Konstantin made some experiments [0].I feel that there is now pretty strong consensus that it would be a good thing, more so than before. Lots of work to get there, and lots of details to be hashed out, but no objections to the idea at a high level.The purpose of this email is to make that silent consensus explicit. If you have objections to switching from the current multi-process architecture to a single-process, multi-threaded architecture, please speak up.For the record, I think this will be a disaster. There is far too much code that will get broken, largely silently, and much of it is not under our control.Noted. Other large projects have gone through this transition. It's not easy, but it's a lot easier now than it was 10 years ago. The platform and compiler support is there now, all libraries have thread-safe interfaces, etc.I don't expect you or others to buy into any particular code change at this point, or to contribute time into it. Just to accept that it's a worthwhile goal. If the implementation turns out to be a disaster, then it won't be accepted, of course. But I'm optimistic.
I don't have enough expertise in this area to comment on if it'd be a "disaster" or not. My zoomed out observations are two-fold:
1. It seems like there's a lack of consensus on which of processes vs. threads yield the best performance benefit, and from talking to folks with greater expertise than me, this can vary between workloads. I believe one DB even gives uses a choice if they want to run in processes vs. threads.
2. While I wouldn't want to necessarily discourage a moonshot effort, I would ask if developer time could be better spent on tackling some of the other problems around vertical scalability? Per some PGCon discussions, there's still room for improvement in how PostgreSQL can best utilize resources available very large "commodity" machines (a 448-core / 24TB RAM instance comes to mind).
I'm purposely giving a nonanswer on whether it's a worthwhile goal, but rather I'd be curious where it could stack up against some other efforts to continue to help PostgreSQL improve performance and handle very large workloads.
Thanks, Jonathan
OpenPGP_signature
Description: OpenPGP digital signature
