On Sunday, July 6, 2014 3:31:43 PM UTC-7, Adrian Mugnolo wrote:
>
> Hi,
>
> I'm seeing this issue with a .paged_each block that used to work. Right
> now I'm using sequel 4.12.0, mysql2 0.3.16.
>
> Tracked the error to mysql2 code related to streaming being enabled. The
> exception reads:
>
> Sequel::DatabaseError: Mysql2::Error: You have already fetched all the
> rows for this query and streaming is true. (to reiterate you must requery).
>
> This is the code fragment:
>
> def import(*args)
> paged_each do |record|
> begin
> record.import(*args)
> rescue => e
> $stderr.puts "#{name}(#{record.pk.inspect}): #{e}"
> end
> end
> end
>
> This code used to work with sequel 4.2.0, mysql2 0.3.13.
>
> Just wondering if this is a known issue or should try to further isolate
> the problem.
>
You didn't provide a backtrace or an SQL log, and the example code you
provided is not self-contained, so this is difficult to diagnose just by
the example you provide. When trying to debug something, you don't want
code that rescues exceptions and doesn't print the backtrace.
The most likely issue is import is issuing a query, and you shouldn't be
trying to issue a query on the same connection while iterating over the
results of another query. However, that issue is not specific to
streaming. If that is the issue, you could try using a separate thread to
force another connection:
Thread.new{record.import(*args)}.join
If that fixes things, then that's the root cause of your issue. If it
worked previously before paged_each used streaming, you were just getting
lucky, as any time you issue a query on the same connection while iterating
over the results of a previous query, you are firmly in the land of
undefined behavior.
The last time this code changed was in 4.9.0, when paged_each was changed
to automatically use streaming. It's possible there are corner cases in
mysql2's streaming support that make it not work with paged_each in all
cases. You can disable the streaming support and see if that fixes it:
Sequel::Mysql2::Dataset::STREAMING_SUPPORTED = false
Even if that does fix it, I'd still like to try to figure out the root
cause of why it doesn't work with streaming.
If the earlier example of using a separate thread doesn't fix things,
please put together a self contained example with a backtrace and SQL log
and I'll try to debug.
Thanks,
Jeremy
--
You received this message because you are subscribed to the Google Groups
"sequel-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sequel-talk.
For more options, visit https://groups.google.com/d/optout.
