On Thu, Jan 15, 2009 at 4:49 PM, Mike Orr <[email protected]> wrote:
> > On Thu, Jan 15, 2009 at 12:49 PM, Joshua Bronson <[email protected]> > wrote: > > > > On Jan 12, 3:11 am, Ben Bangert <[email protected]> wrote: > >> On Jan 11, 2009, at 9:18 PM, Mike Orr wrote: > >> > >> > It does? I've made comparisions between MySQL, SQLite, and Durus, and > >> > MySQL always comes out lowest memory. It never adds more than 20 MB > >> > to the application and 20 MB for the MySQL server, in the same > >> > situations that SQLite or Durus adds 80 MB or even 300 MB. > >> > >> > Which doesn't mean I always use MySQL. Each of them has its own > >> > advantages which sometimes outweigh the memory use. > >> > >> I believe it only occurs if you're forcing it to charset=UTF8. I've > >> seen it a few times, and I believe its an official bug. > http://jjinux.blogspot.com/2008/09/python-debugging-memory-leaks.html > > > > I hadn't heard about this and am now wondering if I should add "? > > charset=utf8&use_unicode=0" to my MySQL connection strings, as advised > > on http://www.sqlalchemy.org/trac/wiki/DatabaseNotes#MySQL. If so, is > > there any migration or anything I'd need to take into account since I > > didn't create or populate it with those flags? > > Well, you know if your program starts increasing memory usage all out > of proportion to the string sizes that should be in memory. > > Also, Ben and I are talking about two different things. Ben is saying > MySQLdb has a memory leak in its Unicode handling. I'm saying MySQLdb > uses *less* memory than SQLite or Durus in my experience. > > If you don't have any concrete problems, I wouldn't bother with the > flags. Just check your data that contains non-ASCII characters and > verify it still contains the correct characters. Also check it in > MySQL's command-line took, and use "show variables like 'c%';" to see > which character set MySQL is using. > mysql> show variables like 'c%'; > +--------------------------+----------------------------+ > | Variable_name | Value | > +--------------------------+----------------------------+ > | character_set_client | utf8 | > | character_set_connection | utf8 | > | character_set_database | utf8 | > | character_set_filesystem | binary | > | character_set_results | utf8 | > | character_set_server | utf8 | > | character_set_system | utf8 | > | character_sets_dir | /usr/share/mysql/charsets/ | > | collation_connection | utf8_general_ci | > | collation_database | utf8_general_ci | > | collation_server | utf8_general_ci | > | completion_type | 0 | > | concurrent_insert | 1 | > | connect_timeout | 5 | > +--------------------------+----------------------------+ > 14 rows in set (0.00 sec) > > The connection character set is what MySQLdb sees and charset= sets. > If any of the other character sets are different, MySQL will convert > characters when writing/reading the table. In the middle is MySQLdb, > which will convert to/from Unicode if use_unicode=1. At the near end > is SQLAlchemy, which will convert to/from Unicode if the engine option > convert_unicode is true (or if a column is defined as UnicodeColumn, > but I don't see why you'd want some columns Unicode and some not). > The SQLAlchemy manual recommends using convert_unicode and not > use_unicode=. > > If you don't set charset in the connection string, the only issue is > whether it defaults to utf8. And for that, you'll have to use the > 'show variables' command to see. Hopefully the default charset was > set to utf8 in my.cnf when MySQL was installed, then you don't have to > worry. But the original default was latin-1. If so, you have some > conversion work to do if you want to make a pure UTF-8 system. > > -- > Mike Orr <[email protected]> > Thanks Mike, that helps a bunch. Fortunately our gentoo servers were set up with utf8. Unfortunately my MacBook is another story, but now I know where to look if I have a problem. (If anybody else who used MySQL's binary installer for 10.5 happens to have changed the default installation to use a my.cnf rather than the compiled-in defaults, I'd be interested to see your config.) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
