Mike, I know that's the *supposed* behaviour. I'm just telling you that
when I have stream_results enabled, I get errors on create_all() /
drop_all() :-)
Cheers,
Richard.
On 04/29/2013 02:06 PM, Michael Bayer wrote:
Well stream_results is only valid with a SELECT statement. To use PG
server side cursors as much as possible, what you need to do is do
create_engine() with server_side_cursors=True which will take effect
for any SELECT statements, but not any
CREATE/DROP/INSERT/UPDATE/DELETE. This is a PG specific feature.
Also the bug seems like it might be limited to the fetchall() and
fetchmany() methods. If I get rows back with fetchone() it appears
things might be OK but am still testing.
On Apr 29, 2013, at 12:59 PM, Richard Gerd Kuesters
<[email protected] <mailto:[email protected]>> wrote:
Well, just to add: if I run my application (using psycopg2) with
stream_results=True, I get some weird errors also; even on
create_all() or drop_all() commands, which I think are pretty basic.
Or is it mixing the commands and not executing in order?
On 04/29/2013 01:55 PM, Michael Bayer wrote:
Well seems like the "stream_results" feature must be extremely
unpopular because I can confirm this behavior even using
connection-local execution options, in 0.7 also. This feature is
tested so will have to see exactly what's up with those tests.
On Apr 29, 2013, at 9:49 AM, Pau Tallada <[email protected]
<mailto:[email protected]>> wrote:
Hi,
I have some very big datasets that we want to process in batches of
reasonable size.
In order to do that, we enable the 'stream_results' execution
option (available in Postgresql) and we use the fetchmany method to
retrieve the records in batches.
With those records we must build a numpy array, but its constructor
complains about the RowProxy wrapper.
So I tried using the cursor directly to retrieve the selected rows,
but the cursor skips the first one if 'stream_results' is enabled.
Let me explain with an example:
engine = create_engine(db_url, execution_options =
{'stream_results' : True})
rs = engine.execute("SELECT * FROM table")
print len(rs.fetchall())
rs = engine.execute("SELECT * FROM table")
print len(rs.cursor.fetchall())
The first one displays *N*, while the second one displays *N-1*.
Any clues? :/
Thank you in advance!
Pau.
--
----------------------------------
Pau Tallada Crespí
Dep. d'Astrofísica i Cosmologia
Port d'Informació Científica (PIC)
Tel: +34 93 586 8233
----------------------------------
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
