On Fri, Feb 1, 2019 at 9:00 AM Robert Haas <robertmh...@gmail.com> wrote: > I don't think we'd be using pqmq here, or shm_mq either, but I think > the bigger issues is that starting a parallel query is already a > pretty heavy operation, and so the added overhead of this is probably > not very noticeable. I agree that it seems a bit expensive, but since > we're already waiting for the postmaster to fork() a new process which > then has to initialize itself, this probably won't break the bank. > What bothers me more is that it's adding a substantial amount of code > that could very well contain bugs to fix something that isn't clearly > a problem in the first place.
I spent most of the last 6 hours writing and debugging a substantial chunk of the code that would be needed. Here's an 0006 patch that adds functions to serialize and restore PartitionDesc in a manner similar to what parallel query does for other object types. Since a PartitionDesc includes a pointer to a PartitionBoundInfo, that meant also writing functions to serialize and restore those. If we want to go this route, I think the next thing to do would be to integrate this into the PartitionDirectory infrastructure. Basically what I'm imagining we would do there is have a hash table stored in shared memory to go with the one that is already stored in backend-private memory. The shared table stores serialized entries, and the local table stores normal ones. Any lookups try the local table first, then the shared table. If we get a hit in the shared table, we deserialize whatever we find there and stash the result in the local table. If we find it neither place, we generate a new entry in the local table and then serialize it into the shard table. It's not quite clear to me at the moment how to solve the concurrency problems associated with this design, but it's probably not too hard. I don't have enough mental energy left to figure it out today, though. After having written this code, I'm still torn about whether to go further with this design. On the one hand, this is such boilerplate code that it's kinda hard to imagine it having too many more bugs; on the other hand, as you can see, it's a non-trivial amount of code to add without a real clear reason, and I'm not sure we have one, even though in the abstract it seems like a better way to go. Still interesting in hearing more opinions. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
0006-Serialize-and-restore-ParitionDesc-PartitionBound.patch
Description: Binary data
