Ok, recently Scott Henson, John Eckersberg, and I were trying to speed up cobbler ... a lot. We got a lot of good things done, including cutting the sync time in half. One thing we also achieved is we found out, that using the python profiler, even though the new yaml implementation is more robust (good, that's the most important thing!), it's also slow... even with libyaml installed. That all being said, the old implementation was also slow. We want /fast/.
Some quick benchmarks on my end show that saving objects using simplejson instead of PyYAML result in 10-12 second times to edit 200 objects versus 28 seconds. As a result of this, I'm going to go ahead and add "serializer_catalog2" to the distribution, though I should warn you ... there's no upgrade path. The way to replicate from catalog to catalog2 is to edit modules.conf on a /new/ cobbler machine and cobbler replicate the data to it (since the wire protocol is exactl y the same). Simplejson offers two output modes -- one is a "pretty printed" mode, this mode is maybe 30% slower. It depends how cleanly readable we need the output to be, but the answer is "not really", since we can always get a pretty printed version of a data structure someone pastebins using ipython if absolutely needed. I am in favor of not using the pretty printed version. Just for kicks, I think I'll benchmark cPickle again -- shelve (which uses things like berkley DB) tends to have issues. We may make the JSON implementation the new default in 1.8, we'll see. --Michael _______________________________________________ cobbler mailing list [email protected] https://fedorahosted.org/mailman/listinfo/cobbler
