On Thu, Jan 15, 2009 at 12:49 PM, Joshua Bronson <[email protected]> wrote: > > On Jan 12, 3:11 am, Ben Bangert <[email protected]> wrote: >> On Jan 11, 2009, at 9:18 PM, Mike Orr wrote: >> >> > It does? I've made comparisions between MySQL, SQLite, and Durus, and >> > MySQL always comes out lowest memory. It never adds more than 20 MB >> > to the application and 20 MB for the MySQL server, in the same >> > situations that SQLite or Durus adds 80 MB or even 300 MB. >> >> > Which doesn't mean I always use MySQL. Each of them has its own >> > advantages which sometimes outweigh the memory use. >> >> I believe it only occurs if you're forcing it to charset=UTF8. I've >> seen it a few times, and I believe its an official bug. >> http://jjinux.blogspot.com/2008/09/python-debugging-memory-leaks.html > > I hadn't heard about this and am now wondering if I should add "? > charset=utf8&use_unicode=0" to my MySQL connection strings, as advised > on http://www.sqlalchemy.org/trac/wiki/DatabaseNotes#MySQL. If so, is > there any migration or anything I'd need to take into account since I > didn't create or populate it with those flags?
Well, you know if your program starts increasing memory usage all out of proportion to the string sizes that should be in memory. Also, Ben and I are talking about two different things. Ben is saying MySQLdb has a memory leak in its Unicode handling. I'm saying MySQLdb uses *less* memory than SQLite or Durus in my experience. If you don't have any concrete problems, I wouldn't bother with the flags. Just check your data that contains non-ASCII characters and verify it still contains the correct characters. Also check it in MySQL's command-line took, and use "show variables like 'c%';" to see which character set MySQL is using. mysql> show variables like 'c%'; +--------------------------+----------------------------+ | Variable_name | Value | +--------------------------+----------------------------+ | character_set_client | utf8 | | character_set_connection | utf8 | | character_set_database | utf8 | | character_set_filesystem | binary | | character_set_results | utf8 | | character_set_server | utf8 | | character_set_system | utf8 | | character_sets_dir | /usr/share/mysql/charsets/ | | collation_connection | utf8_general_ci | | collation_database | utf8_general_ci | | collation_server | utf8_general_ci | | completion_type | 0 | | concurrent_insert | 1 | | connect_timeout | 5 | +--------------------------+----------------------------+ 14 rows in set (0.00 sec) The connection character set is what MySQLdb sees and charset= sets. If any of the other character sets are different, MySQL will convert characters when writing/reading the table. In the middle is MySQLdb, which will convert to/from Unicode if use_unicode=1. At the near end is SQLAlchemy, which will convert to/from Unicode if the engine option convert_unicode is true (or if a column is defined as UnicodeColumn, but I don't see why you'd want some columns Unicode and some not). The SQLAlchemy manual recommends using convert_unicode and not use_unicode=. If you don't set charset in the connection string, the only issue is whether it defaults to utf8. And for that, you'll have to use the 'show variables' command to see. Hopefully the default charset was set to utf8 in my.cnf when MySQL was installed, then you don't have to worry. But the original default was latin-1. If so, you have some conversion work to do if you want to make a pure UTF-8 system. -- Mike Orr <[email protected]> --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
