[web2py] Re: export-table timeout?
On 15 Mar 2013, at 6:45 AM, Jonathan Lundell jlund...@pobox.com wrote:
I'm trying to export a largish (270K records) MySQL table via the grid or
administrative export-csv functions. In both cases, I get an error after a
fairly long wait; I'm assuming it's a connection timeout.
Traceback (most recent call last):
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/main.py,
line 523, in wsgibase
BaseAdapter.close_all_instances('commit')
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/dal.py,
line 447, in close_all_instances
getattr(instance, action)()
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/dal.py,
line 1430, in commit
return self.connection.commit()
File
/home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/contrib/pymysql/connections.py,
line 562, in commit
self.errorhandler(None, exc, value)
File
/home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/contrib/pymysql/connections.py,
line 184, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (2013, 'Lost connection to MySQL server during query')
Should I just increase a timeout value somewhere (and if so, where?), or is
there a better way to do this? FWIW, a gzipped mysqldump of the entire
database is only 22MB, so we're not talking ridiculously large files here.
FWIW: this is a fairly conventional setup, running RHEL6 on a Rackspace cloud
server, with MySQL on a separate virtual machine.
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: export-table timeout?
I'd need to test it again, but when I added excel-like exports my users
were downloading ~50mb of files without hiccups (MSSQL though).
*lost connection during query* seems to point the finger towards mysql not
shipping data fast enough than web2py not fast enough to handle its
serialization. did you try raising the timeout in the driver (or
requesting less data) ?
On Friday, March 15, 2013 3:19:11 PM UTC+1, Jonathan Lundell wrote:
On 15 Mar 2013, at 6:45 AM, Jonathan Lundell jlun...@pobox.comjavascript:
wrote:
I'm trying to export a largish (270K records) MySQL table via the grid or
administrative export-csv functions. In both cases, I get an error after a
fairly long wait; I'm assuming it's a connection timeout.
Traceback (most recent call last):
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/main.py,
line 523, in wsgibase
BaseAdapter.close_all_instances('commit')
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/dal.py,
line 447, in close_all_instances
getattr(instance, action)()
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/dal.py,
line 1430, in commit
return self.connection.commit()
File
/home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/contrib/pymysql/connections.py,
line 562, in commit
self.errorhandler(None, exc, value)
File
/home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/contrib/pymysql/connections.py,
line 184, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (2013, 'Lost connection to MySQL server during query')
Should I just increase a timeout value somewhere (and if so, where?), or
is there a better way to do this? FWIW, a gzipped mysqldump of the entire
database is only 22MB, so we're not talking ridiculously large files here.
FWIW: this is a fairly conventional setup, running RHEL6 on a Rackspace
cloud server, with MySQL on a separate virtual machine.
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
On 15 Mar 2013, at 7:39 AM, Niphlod niph...@gmail.com wrote:
I'd need to test it again, but when I added excel-like exports my users were
downloading ~50mb of files without hiccups (MSSQL though).
lost connection during query seems to point the finger towards mysql not
shipping data fast enough than web2py not fast enough to handle its
serialization. did you try raising the timeout in the driver (or
requesting less data) ?
I haven't tried anything yet.
Re: requesting less data: I could write a custom query function to do that, but
right now I'm just using the existing UI, which in both cases exports the whole
table.
How would I go about raising the timeout in the driver? I looked briefly at
dal.py and it wasn't obvious.
On Friday, March 15, 2013 3:19:11 PM UTC+1, Jonathan Lundell wrote:
On 15 Mar 2013, at 6:45 AM, Jonathan Lundell jlun...@pobox.com wrote:
I'm trying to export a largish (270K records) MySQL table via the grid or
administrative export-csv functions. In both cases, I get an error after a
fairly long wait; I'm assuming it's a connection timeout.
Traceback (most recent call last):
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/main.py,
line 523, in wsgibase
BaseAdapter.close_all_instances('commit')
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/dal.py,
line 447, in close_all_instances
getattr(instance, action)()
File /home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/dal.py,
line 1430, in commit
return self.connection.commit()
File
/home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/contrib/pymysql/connections.py,
line 562, in commit
self.errorhandler(None, exc, value)
File
/home/wupadmin/.virtualenvs/watchup/watchup/web2py/gluon/contrib/pymysql/connections.py,
line 184, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (2013, 'Lost connection to MySQL server during query')
Should I just increase a timeout value somewhere (and if so, where?), or is
there a better way to do this? FWIW, a gzipped mysqldump of the entire
database is only 22MB, so we're not talking ridiculously large files here.
FWIW: this is a fairly conventional setup, running RHEL6 on a Rackspace cloud
server, with MySQL on a separate virtual machine.
--
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
you could set some filters and with the frid it should export only the
records shown, not the whole table
PS: passing driver_args = {} should do the job with pymysql, but I don't
see any forums showing that that kind of error is driver-dependant on some
kind of timeout parameter (as is in sqlite) seems a mysql configuration
directive (meaning my.cnf, mysql.ini, etc)...
*wait_timeout = *
*connect_timeout =*
or more probably
*net_read_timeout*
cfr. http://dev.mysql.com/doc/refman/5.0/en/error-lost-connection.html
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
On 15 Mar 2013, at 8:15 AM, Niphlod niph...@gmail.com wrote:
you could set some filters and with the frid it should export only the
records shown, not the whole table
I think I'll write a script to parse the database dump into a csv file instead;
it's a lot quicker.
But this seems like a bug to me: at least in the admin app, a table export
ought to Just Work; it's pretty trivial, after all, even if it's large.
PS: passing driver_args = {} should do the job with pymysql, but I don't see
any forums showing that that kind of error is driver-dependant on some kind
of timeout parameter (as is in sqlite) seems a mysql configuration
directive (meaning my.cnf, mysql.ini, etc)...
wait_timeout =
connect_timeout =
or more probably
net_read_timeout
cfr. http://dev.mysql.com/doc/refman/5.0/en/error-lost-connection.html
--
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
In my experience recently I had many problems with pymysql, so I switch to
mysqldb using dal.
El 15/03/2013 13:18, Jonathan Lundell jlund...@pobox.com escribió:
On 15 Mar 2013, at 8:15 AM, Niphlod niph...@gmail.com wrote:
you could set some filters and with the frid it should export only the
records shown, not the whole table
I think I'll write a script to parse the database dump into a csv file
instead; it's a lot quicker.
But this seems like a bug to me: at least in the admin app, a table export
ought to Just Work; it's pretty trivial, after all, even if it's large.
PS: passing driver_args = {} should do the job with pymysql, but I don't
see any forums showing that that kind of error is driver-dependant on some
kind of timeout parameter (as is in sqlite) seems a mysql configuration
directive (meaning my.cnf, mysql.ini, etc)...
*wait_timeout = *
*connect_timeout =*
or more probably
*net_read_timeout*
cfr. http://dev.mysql.com/doc/refman/5.0/en/error-lost-connection.html
--
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an
email to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
On 15 Mar 2013, at 3:44 PM, Martín Mulone mulone.mar...@gmail.com wrote:
In my experience recently I had many problems with pymysql, so I switch to
mysqldb using dal.
I'll give it a try. A question, though. The book suggests:
from gluon.dal import MySQLAdapter
MySQLAdapter.driver = mysqldb
But just what is the symbol mysqldb? And how does it relate to the driver names
in gluon.dal?
import MySQLdb
DRIVERS.append('MySQL(MySQLdb)')
...
class MySQLAdapter(BaseAdapter):
drivers = ('MySQLdb','pymysql')
El 15/03/2013 13:18, Jonathan Lundell jlund...@pobox.com escribió:
On 15 Mar 2013, at 8:15 AM, Niphlod niph...@gmail.com wrote:
you could set some filters and with the frid it should export only the
records shown, not the whole table
I think I'll write a script to parse the database dump into a csv file
instead; it's a lot quicker.
But this seems like a bug to me: at least in the admin app, a table export
ought to Just Work; it's pretty trivial, after all, even if it's large.
PS: passing driver_args = {} should do the job with pymysql, but I don't see
any forums showing that that kind of error is driver-dependant on some kind
of timeout parameter (as is in sqlite) seems a mysql configuration
directive (meaning my.cnf, mysql.ini, etc)...
wait_timeout =
connect_timeout =
or more probably
net_read_timeout
cfr. http://dev.mysql.com/doc/refman/5.0/en/error-lost-connection.html
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
On Friday, 15 March 2013 19:31:07 UTC-5, Jonathan Lundell wrote:
On 15 Mar 2013, at 3:44 PM, Martín Mulone mulone...@gmail.comjavascript:
wrote:
In my experience recently I had many problems with pymysql, so I switch to
mysqldb using dal.
I'll give it a try. A question, though. The book suggests:
from gluon.dal import MySQLAdapter
MySQLAdapter.driver = mysqldb
should be
MySQLAdapter.driver = 'mysqldb'
Basically is a list of names of supported drivers MySQLAdapter.drivers and
MySQLAdapter.driver
is the name of the selected one. Must be installed.
But just what is the symbol mysqldb? And how does it relate to the driver
names in gluon.dal?
import MySQLdb
DRIVERS.append('MySQL(MySQLdb)')
...
class MySQLAdapter(BaseAdapter):
drivers = ('MySQLdb','pymysql')
El 15/03/2013 13:18, Jonathan Lundell jlun...@pobox.com javascript:
escribió:
On 15 Mar 2013, at 8:15 AM, Niphlod nip...@gmail.com javascript:
wrote:
you could set some filters and with the frid it should export only the
records shown, not the whole table
I think I'll write a script to parse the database dump into a csv file
instead; it's a lot quicker.
But this seems like a bug to me: at least in the admin app, a table
export ought to Just Work; it's pretty trivial, after all, even if it's
large.
PS: passing driver_args = {} should do the job with pymysql, but I don't
see any forums showing that that kind of error is driver-dependant on some
kind of timeout parameter (as is in sqlite) seems a mysql configuration
directive (meaning my.cnf, mysql.ini, etc)...
*wait_timeout = *
*connect_timeout =*
or more probably
*net_read_timeout*
cfr. http://dev.mysql.com/doc/refman/5.0/en/error-lost-connection.html
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
Can you tel us something about problems. We are using it to check tests
with mysql and some are failing on travis.ci we are trying to figure out
why. Could it be the driver?
On Friday, 15 March 2013 17:44:54 UTC-5, Martin.Mulone wrote:
In my experience recently I had many problems with pymysql, so I switch to
mysqldb using dal.
El 15/03/2013 13:18, Jonathan Lundell jlun...@pobox.com javascript:
escribió:
On 15 Mar 2013, at 8:15 AM, Niphlod nip...@gmail.com javascript:
wrote:
you could set some filters and with the frid it should export only the
records shown, not the whole table
I think I'll write a script to parse the database dump into a csv file
instead; it's a lot quicker.
But this seems like a bug to me: at least in the admin app, a table
export ought to Just Work; it's pretty trivial, after all, even if it's
large.
PS: passing driver_args = {} should do the job with pymysql, but I don't
see any forums showing that that kind of error is driver-dependant on some
kind of timeout parameter (as is in sqlite) seems a mysql configuration
directive (meaning my.cnf, mysql.ini, etc)...
*wait_timeout = *
*connect_timeout =*
or more probably
*net_read_timeout*
cfr. http://dev.mysql.com/doc/refman/5.0/en/error-lost-connection.html
--
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an
email to web2py+un...@googlegroups.com javascript:.
For more options, visit https://groups.google.com/groups/opt_out.
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: export-table timeout?
On 15 Mar 2013, at 7:34 PM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: On Friday, 15 March 2013 19:31:07 UTC-5, Jonathan Lundell wrote: On 15 Mar 2013, at 3:44 PM, Martín Mulone mulone...@gmail.com wrote: In my experience recently I had many problems with pymysql, so I switch to mysqldb using dal. I'll give it a try. A question, though. The book suggests: from gluon.dal import MySQLAdapter MySQLAdapter.driver = mysqldb should be MySQLAdapter.driver = 'mysqldb' Basically is a list of names of supported drivers MySQLAdapter.drivers and MySQLAdapter.driver is the name of the selected one. Must be installed. See my later message to the dev list, though. There is no driver name 'mysqldb', though there is one 'MySQLdb'. And the adapter code already has two different mechanisms for specifying its name at DAL init time. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
