Sean,
Absolutely agree with you. It's not the same to execute query and get plain text, and execute query and get hierarchy of python objects. Plus I disagree when I hear that SQLAlchemy is slow. It's slow when you are using it wrong. Like in Nova Scheduler [1] we were fetching full 3 tables with JOIN. Which produce much more results from DB (in bytes and rows) then just make 3 separated selects and then join it by hand. We should stop using next phrases: 1) python is slow 2) mysql is slow 3) sqlalchemy is slow 4) hardware is slow [2] And start using these phrase: 1) Algorithms that we are using are bad 2) Architecture solutions that we are using are bad And start thinking about how to improve them. [1] https://review.openstack.org/#/c/43151/ [2] http://en.wikipedia.org/wiki/Buran_(spacecraft) Best regards, Boris Pavlovic On Fri, Mar 21, 2014 at 3:04 PM, Sean Dague <[email protected]> wrote: > On 03/20/2014 06:18 PM, Joe Gordon wrote: > > > > > > > > On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko > > <[email protected] <mailto:[email protected]>> wrote: > > > > Hello, > > > > We've done some profiling and results are quite interesting: > > during 1,5 hour ceilometer inserted 59755 events (59755 calls to > > record_metering_data) > > this calls resulted in total 2591573 SQL queries. > > > > And the most interesting part is that 291569 queries were ROLLBACK > > queries. > > We do around 5 rollbacks to record a single event! > > > > I guess it means that MySQL backend is currently totally unusable in > > production environment. > > > > > > It should be noticed that SQLAlchemy is horrible for performance, in > > nova we usually see sqlalchemy overheads of well over 10x (time > > nova.db.api call vs the time MySQL measures when slow log is recording > > everything). > > That's not really a fair assessment. Python object inflation takes time. > I do get that there is SQLA overhead here, but even if you trimmed it > out you would not get the the mysql query time. > > That being said, having Ceilometer's write path be highly tuned and not > use SQLA (and written for every back end natively) is probably appropriate. > > -Sean > > -- > Sean Dague > Samsung Research America > [email protected] / [email protected] > http://dague.net > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
