Hi At 12:23 PM 2004/10/29, Alexander Malysh wrote:
Nisan Bloch wrote: >>1) don't use dict if you don't know how many items will be stored (dict >>filled to more as 80% works very slow). Here we should use something like >>red-black-tree. > > > Or just change the hash function the dict code uses.
nope, hash function has nothing todo with this issue. Hashtables generally doesn't work well when filled above 80%. So you have only one option: dynamicaly size change (that is very very costly)
One of the main reasons the hash tables fill up like they do in the Kannel is that the current octstr_hash_key doesnt do a nice spread, so we end up using very few of the "slots" in the hash table, and hence the lists get very big. This causes the slowdown.
While we are on the Dict code, I am using a modified version of the List code - Currently the list code is intended for a consumer/producer model, and as such all appends to the list generate a pthread_cond_signal. We have a stripped down ver that we use in the Dict and similar code.
> uuid keys: new hash +-9secs; current 75secs > int keys: new hash +-5secs; current +-11mins
which algo are you using here?
(code attached in previous email)
http://burtleburtle.net/bob/hash/doobs.html http://burtleburtle.net/bob/hash/
Nisan
> > > I can pass on the dict patches to implement this. > > Nisan >>2) don't call any functions in signal handler that are not signal safe >>(you call gwthread_wakeup_all()) >>3) check for 'file != NULL' in 'int store_save(Msg *msg)'. Because file != >>NULL is our marker for enabled store-file support (store-file support is >>optional). >> >> >>Kalle Marjola wrote: >> >> > Attached a patch (against CVS) that greatly improves store-file: >> > It uses Dict instead of List to save messages to memory. >> > Moreover, acknowledged messages are immediately cleaned from >> > memory, instead of being done in a background thread (that thread >> > simple dumps current state to file to reduce startup-times). >> > And, above this all, now it is possible to define the frequency >> > how often this 'cleanup' is done.. >> > >> > Thus it is far more powerful if queues are created, and I >> > greatly encourage all kannel instances to use it, at least >> > after it has gained adequate extra testing (I have done some >> > tests but more are naturally needed) >> > >> > As part of the patch, smskannel.conf is updated to enable >> > store-file - I know that lots of Kannel installation are based >> > on smskannel.conf which is then modified... >> > >> > Comments? >> > I can put this into CVS immediately after Stipe puts out 1.4.0,.. >> > >> >>-- >>Thanks, >>Alex > > > -------------------------------- > www.clickatell.com > Any message, anywhere > Phone: +27 21 9487150
-- Thanks, Alex
-------------------------------- www.clickatell.com Any message, anywhere Phone: +27 21 9487150
