So to implement recursive queries I think what we need is a memoizing node like Materialize which allows multiple simultaneous readers.
Looking into how to implement this I find that the read position of a Materialize node is actually implemented directly in tuplestore.c. That means that tuplestore.c would have to grow a bit to allow for an array of structs that contain current, readpos_file, readpos_offset, markpos_current, markpos_file, and markpos_offset. An alternative I've been thinking about is creating a kind of slave_tuplestore. It would keep its own readpos etc, but pass all actual operations through to another tuplestore. I don't think it would pay for itself though. The complexity involved especially around when arrays are grown and dumped to disk would be greater than the complexity saved in the regular codepath. Secondly, it would be valuable for recursive queries to be able to throw away old tuples when all the marks and read positions have passed them (if they're not random-access of course). This is the same feature that Simon suggested for merge joins. I'm thinking the way to do that is to keep an array of arrays instead of a single array. Whenever (all) the mark and read position(s) advances over the end of an array we can free it. Alternatively whenever it comes time to grow the array we could check whether older tuples aren't needed any more and memmove the later tuples back to the start of the array instead of growing the array. That would be simpler but it would be more expensive to do the memmove. Often there would be very few tuples moving though, and repalloc often does a memcpy anyways, so perhaps it wouldn't really be a problem. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---------------------------(end of broadcast)--------------------------- TIP 4: Have you searched our list archives? http://archives.postgresql.org
