On 10/8/06, Brian Safford <[EMAIL PROTECTED]> wrote: > Please keep large implementations in mind while engineering this ... > once I can get PyYIMt patched to work with Jabber XCP, I have about > 12,000 users (3,000 concurrent) to throw at it ...
Yea, I'd prefer to keep the data file small to make the accesses faster. (Less data means faster right?) Avatar access is relatively seldom, so moving the data out of 'primary' database can only help there. btw: I think you promised to send the XCP patches in this direction :-P -- - Norman Rasmussen - Email: [EMAIL PROTECTED] - Home page: http://norman.rasmussen.co.za/ From [EMAIL PROTECTED] Sun Oct 8 14:58:18 2006 From: [EMAIL PROTECTED] (Daniel Henninger) Date: Sun Oct 8 14:58:30 2006 Subject: [py-transports] PyYIMt avatars In-Reply-To: <[EMAIL PROTECTED]> References: <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> Message-ID: <[EMAIL PROTECTED]> Hi Norman! While it is a lot more work, most of that work you can probably steal from PyAIMt/ICQt/MSNt. (maybe, forget how much if it is twisted dependent) Anyway, the single file will end up becoming a problem quick on some systems. Some Solaris's have cap of 2 gig per single file, for example. Keep in mind that once you start doing it in multiple files you'll run into the same problems we did, where you end up needing to create different subdirectories. (too many files in one directory causes problems for some systems... I think this applies to all systems just some run into the limit quicker than others) Daniel On Oct 8, 2006, at 10:36 AM, Brian Safford wrote: > Please keep large implementations in mind while engineering this ... > once I can get PyYIMt patched to work with Jabber XCP, I have about > 12,000 users (3,000 concurrent) to throw at it ... > > Thanks, > Brian Safford > EDS > > On Sun, 2006-10-08 at 15:55 +0200, Norman Rasmussen wrote: >> I'm interested in what people's thoughts are on the following: >> >> Currently the PyYIMt subscription data is stored (all in one file) >> like this: >> >> [EMAIL PROTECTED]: >> userrname: yuser1 >> password: ypassword1 >> subscribed: true >> avatar: >> yusera: (sha1hash, avatarimagedata) >> yuserb: (sha1hash, avatarimagedata) >> yuserc: (sha1hash, avatarimagedata) >> [EMAIL PROTECTED]: >> userrname: yuser2 >> password: ypassword2 >> subscribed: true >> avatar: >> yuserc: (sha1hash, avatarimagedata) >> yuserd: (sha1hash, avatarimagedata) >> yusere: (sha1hash, avatarimagedata) >> >> Now obviously this means that the single file gets fairly big with >> all >> the avatars, and that if multiple transport users have the same >> contacts, or if multiple contacts have the same avatar, then that >> data >> is stored multiple times. >> >> Does it make sense to extract the avatar out of the data file, and >> store it like the other transports? If so, does it get stored like >> pymsnt (newer - global for the whole transport), or pyaimt/pyictq >> (older - set of avatars per copy of transport). I think that storing >> the avatars on the filesystem as individual files is a bonus, because >> then the admin can use standard tools to archive old/unused data. >> >> What are people's thoughts on converting the entire file to the file >> system based format (xml files in sha1'ed folders) - btw: this is a >> lot more work than just extracting the avatar image data :-P >> > > _______________________________________________ > py-transports mailing list > [email protected] > http://www.modevia.com/cgi-bin/mailman/listinfo/py-transports
