Dear All,
I wish to share some of my findings (may be useful to someone).
And one question (at the bottom of this post).
Task: Displaying a report with a large amount of data returned from
controller.
A resultset is fetched from MySQL
custdata = db.executesql(qry, as_dict=True)
There are 10,000+ records in it.
----------------------------------------------
Earlier, I was parsing 'custdata' using 'for' loop---
----------------------------------------------
for i in custdata:
mystr2 = ''.join([mystr2, '<tr><td>' , str(i[0]) , '</td>'])
mystr2 = ''.join([mystr2, '<td>' , str(i[1]) , '</td>'])
mystr2 = ''.join([mystr2, '<td>' , str(i[2]) , '</td>'])
......more omitted......
mystr2 = ''.join([mystr2, '<td>' , str(i[12]) , '</td></tr>'])
mystr3 = '</tbody></table>'
mystr = ''.join([mystr1, mystr2, mystr3])
----------------------------------------------
It took 175 seconds to process the data.
1) Using 'for' loop is slowing down the processing.
2) Evaluating ''.join everytime in 'for' loop was costly.
----------------------------------------------
Now, I parsed 'custdata' like ---
----------------------------------------------
1) Used list comprehension instead of 'for' loop
2) Used jn=''.join (i.e. local variable instead of global variable)
jn = ''.join
lst = [ jn(['<tr><td>' , str(i[0]) , '</td>',
'<td>' , str(i[1]) , '</td>',
'<td>' , str(i[2]) , '</td>',
...... more omitted......]) for i in res]
mystr=jn(lst)
----------------------------------------------
The performance boost is amazing......
It took ONLY 0.078 seconds
----------------------------------------------
My Question is this---
----------------------------------------------
If I send the big string output (which contains html code) to a View
from a Controller like this---
return dict(mystr=mystr)
and render it like this---
{{=XML(mystr)}}
it takes again 1 minute to display the plain html data.
(not to speak of using dataTables; it haults the task).
For testing purpose, I saved the output string in .html file like
this---
flnm=open('myfile.html', 'w')
flnm.writelines(mystr)
When I loaded the .html file in browser (outside of web2py), it loads
within 5 seconds.
In Web2Py, how do I render it quickly in View?
Thanks,
Vineet
