On Sun, Jul 08, 2018 at 04:47:47PM +0200, Christoph Zwerschke wrote:
> [...] as we can make the query itself iterable; it would then return tuples.
> dictiter() could become as_dicts, namediter() could become with_names, used
> like so:
[...]
> for row in query.with_names:
> print row.id, row.name
Am 08.07.2018 um 18:29 schrieb D'Arcy Cain:
> I don't see why this at least couldn't work without "with_names".
On Sun, Jul 08, 2018 at 07:11:41PM +0200, Christoph Zwerschke wrote:
> Sure, we could return named tuples by default, or even a hybrid object that
> could also be accessed as dict. Then we wouldn't need that distinction
> between various flavors of results. Maybe it would be the most elegant
> solution. But of course there would be some memory overhead compared with
> plain tuple/dict objects.
Memory use should be a nonissue for iterators, right ? Right now, published
pygres stores an arbitrarily large result set in libpq as native("C") objects
*and* converts to python, which itself ends up taking ~4x as much RAM. In my
patch, iterators fix the 4x part and only convert one row (or some small batch
size) to python objects at a time. But my patch doesn't use PG cursors, so I
don't think there's a significant argument for converting to only one of three
result types - at least not for savings in RAM.
I have patches to use PG cursors too..but they're uglier and more invasive and
we agreed that it's worth publishing a version with "soft" cursors first. It
may turn out that jury-rigging PG cursors onto the classic interface isn't
worth it.
Justin
_______________________________________________
PyGreSQL mailing list
PyGreSQL@vex.net
https://mail.vex.net/mailman/listinfo.cgi/pygresql
