Hello, > Query fewer objects from the database. Make sure you don't store lots > of tiny persistent objects in the database, I'd aim for storing data > in chunks of 8-32kb or use blobs for larger objects. Remember that > ZODB is a key/value storage for the most part. Model your data > accordingly.
That's hard to do for a project that is already 8 or 9 years old, as you can see in the attached file, we've got have many cases that fall outside your limits. I've noticed, for instance, that pages that involve the loading of >200 MaKaC.review.Abstract objects have an awful performance record (maybe because we then load for each object a handful of other referenced persitent objects). > Before you do that, you might consider switching to RelStorage, which > already has a memcached caching layer in addition to the connection > caches. But isn't RelStorage supposed be slower than FileStorage/ZEO? > But remember that throwing more caches at the problem isn't a > solution. It's likely the way you store or query the data from the > database that's not optimal. I agree, many things could be improved regarding the data structures we use. However, it is also true that we have a large number of objects that are rarely changed, and that there is no need to fetch from the DB if we can keep them in memory. > The ZEO server needs almost no CPU power, except for garbage > collection and packing. During normal operations the CPU speed should > be irrelevant. Yes, it is consistent with our load average history. It slightly increases during DB packing, but otherwise stays around 1. Thanks a lot for your answers. Cheers, Pedro -- José Pedro Ferreira Software Developer, Indico Project http://indico-software.org +-----------+ + '``'--- `+ CERN - European Organization for Nuclear Research + |CERN| / + 1211 Geneve 23, Switzerland + ..__. \. + IT-UDS-AVC + \\___.\ + Office: 513-R-0042 + / + Tel. +41227677159 +-----------+
Module.ClassName Percentage Min Max Size Instances ______________________________________________________________________________________________________________________________________ MaKaC.webinterface.displayMgr.SystemLink | 13.7365089336% | 165B | 551B | 1753.1Mb | 7108834 MaKaC.conference.Contribution | 7.33041346279% | 954B | 12.4Kb | 935.5Mb | 719579 MaKaC.accessControl.AccessController | 5.33470736765% | 159B | 1.7Kb | 680.8Mb | 2699638 MaKaC.conference.Conference | 4.28939314997% | 1.7Kb | 75.6Kb | 547.4Mb | 165507 MaKaC.common.log.ActionLogItem | 4.17241315243% | 215B | 9.9Kb | 532.5Mb | 1298923 BTrees.OOBTree.OOBucket | 3.35268430987% | 51B | 1.2Mb | 427.9Mb | 237004 MaKaC.common.Counter.Counter | 3.23618867689% | 59B | 70B | 413.0Mb | 7339189 MaKaC.conference.LocalFile | 2.95354653864% | 337B | 7.7Kb | 376.9Mb | 994222 MaKaC.registration.PersonalDataFormItem | 1.99248960658% | 116B | 162B | 254.3Mb | 1951216 MaKaC.conference.ContributionParticipation | 1.52190679944% | 184B | 1.9Kb | 194.2Mb | 739925 MaKaC.common.indexes.EmailIndex | 1.36666809146% | 2.1Mb | 2.1Mb | 174.4Mb | 92 MaKaC.registration.RegistrationForm | 1.30462205201% | 824B | 6.1Kb | 166.5Mb | 162642 MaKaC.conference.Slides | 1.13426433984% | 291B | 1.8Kb | 144.8Mb | 386587 persistent.mapping.PersistentMapping | 1.12364719117% | 57B | 12.4Kb | 143.4Mb | 847891 MaKaC.review.AbstractMgr | 1.10508750233% | 729B | 6.6Kb | 141.0Mb | 162620 MaKaC.conference.ContribStatusSch | 0.903904932477% | 170B | 195B | 115.4Mb | 648183 MaKaC.common.log.EmailLogItem | 0.898982684087% | 295B | 16.0Kb | 114.7Mb | 75793 MaKaC.contributionReviewing.Review | 0.887749184867% | 453B | 453B | 113.3Mb | 262267 MaKaC.conference.ReportNumberHolder | 0.878300608667% | 117B | 251B | 112.1Mb | 984754 MaKaC.registration.AccommodationType | 0.876478211284% | 142B | 438B | 111.9Mb | 488509 MaKaC.schedule.ContribSchEntry | 0.873561973452% | 165B | 228B | 111.5Mb | 558052 MaKaC.webinterface.displayMgr.Menu | 0.866853068433% | 357B | 1.3Kb | 110.6Mb | 190541 MaKaC.conference.Session | 0.809584603165% | 894B | 16.6Kb | 103.3Mb | 83969 MaKaC.registration.MiscellaneousInfoSimpleItem | 0.79980971978% | 216B | 12.8Kb | 102.1Mb | 315249 MaKaC.user.TimedLinkedEvents | 0.747063696646% | 80B | 835.8Kb | 95.3Mb | 16930 [..]binterface.session.sessionManagement.PSession | 0.708889402539% | 299B | 364B | 90.5Mb | 299841 persistent.list.PersistentList | 0.665464072002% | 51B | 2.3Kb | 84.9Mb | 1247969 MaKaC.common.indexes.SurNameIndex | 0.658604230855% | 1.1Mb | 1.1Mb | 84.1Mb | 88 MaKaC.conference.Material | 0.658030032432% | 286B | 4.5Kb | 84.0Mb | 216740 BTrees._OOBTree.OOBTree | 0.603115061848% | 30B | 11.1Kb | 77.0Mb | 1353388 MaKaC.registration.SocialEventForm | 0.578302795899% | 254B | 1.6Kb | 73.8Mb | 162620 MaKaC.webinterface.displayMgr.ConfDisplayMgr | 0.556053343056% | 257B | 665B | 71.0Mb | 162000 MaKaC.user.Avatar | 0.540621497042% | 306B | 442.2Kb | 69.0Mb | 47783 MaKaC.contributionReviewing.ReviewManager | 0.535484012045% | 273B | 316B | 68.3Mb | 262267 MaKaC.common.indexes.OrganisationIndex | 0.51419663037% | 919.8Kb | 921.5Kb | 65.6Mb | 82 MaKaC.conference.SubContribution | 0.512961328868% | 391B | 20.6Kb | 65.5Mb | 106606 MaKaC.reviewing.ConferenceReview | 0.505883194879% | 1004B | 1011B | 64.6Mb | 67439 MaKaC.registration.AccommodationForm | 0.505504386222% | 336B | 2.4Kb | 64.5Mb | 162617 MaKaC.common.indexes.NameIndex | 0.504083259697% | 833.3Kb | 834.5Kb | 64.3Mb | 88 MaKaC.paperReviewing.ConferencePaperReview | 0.492319066691% | 1.4Kb | 1.8Kb | 62.8Mb | 46735 MaKaC.common.log.LogHandler | 0.489389729645% | 125B | 230.2Kb | 62.5Mb | 161374 BTrees.OOBTree.OOBTree | 0.488447475429% | 29B | 15.1Kb | 62.3Mb | 1127411 MaKaC.evaluation.Evaluation | 0.484225493257% | 429B | 5.2Kb | 61.8Mb | 135366 MaKaC.contributionReviewing.RefereeJudgement | 0.432548703775% | 214B | 279B | 55.2Mb | 262267 MaKaC.contributionReviewing.EditorJudgement | 0.43058899255% | 213B | 278B | 55.0Mb | 262267 MaKaC.participant.Participation | 0.422333266833% | 326B | 2.9Kb | 53.9Mb | 160734 MaKaC.review.AbstractField | 0.373892633417% | 140B | 677B | 47.7Mb | 300242 MaKaC.registration.Registrant | 0.372431476949% | 509B | 5.8Kb | 47.5Mb | 55948 MaKaC.badge.BadgeTemplateManager | 0.372123036404% | 337B | 616B | 47.5Mb | 131103 MaKaC.conference.SessionSlot | 0.351320325664% | 465B | 835B | 44.8Mb | 88288 BTrees.OOBTree.OOSet | 0.32404588564% | 31B | 14.6Kb | 41.4Mb | 400049 MaKaC.schedule.ConferenceSchedule | 0.316500581209% | 197B | 3.1Kb | 40.4Mb | 138328 MaKaC.poster.PosterTemplateManager | 0.311345393964% | 344B | 705B | 39.7Mb | 121008 MaKaC.review.Abstract | 0.283116332811% | 914B | 15.0Kb | 36.1Mb | 15461 MaKaC.conference.DeletedObject | 0.278782742044% | 234B | 338B | 35.6Mb | 132081 MaKaC.conference.ConferenceChair | 0.278144422481% | 170B | 1.1Kb | 35.5Mb | 138480 MaKaC.common.indexes.StatusIndex | 0.276938315862% | 516.6Kb | 517.4Kb | 35.3Mb | 79 MaKaC.schedule.LinkedTimeSchEntry | 0.276078266128% | 167B | 277B | 35.2Mb | 176351 BTrees.IOBTree.IOBucket | 0.276075568572% | 53B | 4.7Kb | 35.2Mb | 39091 MaKaC.registration.ReasonParticipationForm | 0.275844333438% | 125B | 1.3Kb | 35.2Mb | 162615 MaKaC.conference.ContribSchEntry | 0.273799324134% | 164B | 232B | 34.9Mb | 169253 MaKaC.plugins.Collaboration.base.CSBookingManager | 0.261076444981% | 272B | 757B | 33.3Mb | 124660 MaKaC.registration.MiscellaneousInfoGroup | 0.258388789563% | 221B | 664B | 33.0Mb | 105350 MaKaC.conference.AcceptedContribution | 0.257560198866% | 1.0Kb | 15.4Kb | 32.9Mb | 11621 MaKaC.registration.SessionsForm | 0.253256161661% | 180B | 935B | 32.3Mb | 162613 MaKaC.conference.AuthorIndex | 0.23525881993% | 87B | 150B | 30.0Mb | 326067 MaKaC.common.pendingQueues.ConfPendingQueuesMgr | 0.229919929841% | 188B | 8.3Kb | 29.3Mb | 162534 MaKaC.badge.BadgePDFOptions | 0.222809462537% | 289B | 337B | 28.4Mb | 91805 MaKaC.participant.Participant | 0.221962168285% | 278B | 648B | 28.3Mb | 77802 indico.ext.livesync.base.ActionWrapper | 0.215379555236% | 186B | 260B | 27.5Mb | 139155 BTrees._OOBTree.OOBucket | 0.212283716937% | 96B | 50.0Kb | 27.1Mb | 44251 MaKaC.registration.PersonalData | 0.206553613914% | 170B | 170B | 26.4Mb | 162609 MaKaC.review.AbstractFieldsMgr | 0.202769726862% | 54B | 323B | 25.9Mb | 150339 MaKaC.conference._PrimAuthIdx | 0.175618006974% | 144B | 161B | 22.4Mb | 162608 MaKaC.registration.FurtherInformationForm | 0.174282088859% | 115B | 1.4Kb | 22.2Mb | 162621 MaKaC.review._PrimAuthIdx | 0.165897179332% | 136B | 157B | 21.2Mb | 162607 MaKaC.review.Author | 0.163428892788% | 184B | 2.1Kb | 20.9Mb | 78702 MaKaC.review.SubmissionNotification | 0.161561391664% | 74B | 304B | 20.6Mb | 161458 MaKaC.registration.Notification | 0.157797112334% | 70B | 3.6Kb | 20.1Mb | 162688 MaKaC.webinterface.displayMgr.TickerTape | 0.156141044337% | 129B | 857B | 19.9Mb | 161862 MaKaC.conference.SubContribParticipation | 0.151651309141% | 185B | 496B | 19.4Mb | 75876 [..]ins.Collaboration.EVO.collaboration.CSBooking | 0.148266488573% | 1.9Kb | 4.2Kb | 18.9Mb | 8519 MaKaC.conference.SubmitterIndex | 0.139422299515% | 91B | 122B | 17.8Mb | 162609 MaKaC.webinterface.displayMgr.Format | 0.136017482656% | 98B | 118B | 17.4Mb | 161897 MaKaC.conference.Link | 0.131640417144% | 160B | 2.6Kb | 16.8Mb | 67239 MaKaC.conference.CustomLocation | 0.129713906039% | 74B | 1.2Kb | 16.6Mb | 205868 MaKaC.conference.Minutes | 0.127516518423% | 302B | 6.5Kb | 16.3Mb | 42516 MaKaC.conference.BOAConfig | 0.125166655353% | 100B | 880B | 16.0Mb | 162609 MaKaC.review._AbstractParticipationIndex | 0.120944949663% | 99B | 129B | 15.4Mb | 162609 MaKaC.schedule.SlotSchedule | 0.114537296686% | 192B | 2.0Kb | 14.6Mb | 62680 MaKaC.conference.SessionChair | 0.102154421729% | 170B | 534B | 13.0Mb | 53515 MaKaC.conference.TCIndex | 0.101505932154% | 83B | 113B | 13.0Mb | 162612 MaKaC.conference.SCIndex | 0.10150460953% | 83B | 113B | 13.0Mb | 162609 MaKaC.schedule.SessionSchedule | 0.100569738099% | 191B | 840B | 12.8Mb | 57430 MaKaC.schedule.BreakTimeSchEntry | 0.0951844664785% | 232B | 2.3Kb | 12.1Mb | 36809 MaKaC.paperReviewing.Status | 0.0911314813598% | 83B | 93B | 11.6Mb | 140189 MaKaC.webinterface.displayMgr.ImagesManager | 0.0807878170819% | 197B | 820B | 10.3Mb | 54793 MaKaC.conference.CustomRoom | 0.0795048637309% | 47B | 183B | 10.1Mb | 182436 MaKaC.webinterface.displayMgr.StyleManager | 0.0741101170368% | 136B | 313B | 9.5Mb | 72824 BTrees.IOBTree.IOBTree | 0.066319222884% | 29B | 21.0Kb | 8.5Mb | 85500
_______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
