Im using a thread to import some rows from a csvfile into the database
(MySQL).
The problem is that the request closes the connection while the thread is
inserting records and i have to restart the server in order to clean up the
hanged thread.
So is there anyway to use DAL with an independent db connection within that
thread?
Calling mythread.join() works but doesnt release the request until is done
and some servers like pythonanyqhere has a timeout for each request.
I cant use the scheduler on my server, it hangs itself after some time and
i have to restart it manually. Threads looks like the easiest choice and
subclassing is very convinient to report percentages, statistics and such.
Heres is some pseudocode im using:
import threading, csv
class importcsv(threading.Thread):
percentage = 0
total_rows = 0
def __init__(self, csvfile, has_headers, ...):
threading.Thread.__init__(self)
self.name = "user-1"
self.daemon = False
self.csvfile = csvfile
...
def run(self):
rows = ... #extract rows from the file
self.total_rows = len(rows)
#process rows
for i, r in enumerate(rows):
inserdata = {....} #row converted to dictionary
db.mytable.insert(**insertdata)
self.percentage = 100 * (i+1) / total_rows
db.commit()
@auth.requires_login()
def index()
threads = [t for t in threading.enumerate() if t.name == "user-%s" %
auth.user.id]
running = len(threads)
if not running:
#display a form to upload the csv file
form = SQLFORM.factory(
#some fields
#csvfile, has_header, column_separator, ...
)
if form.accepted:
job = importcsv(csvfile, has_headers, ...)
else:
#display the percentage done
percentage = threads[0].percentage
return locals()
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
