On Thu, Nov 01, 2012 at 22:34 +0900, Alan Busby wrote: > On Thu, Nov 1, 2012 at 8:27 PM, Wolodja Wentland <babi...@gmail.com> wrote:
> > Oh, fold-into-map and fold-into-map-with would be wonderful and I tried to > > implement the former along the lines of fold-into-vec, but the performance > > was > > abysmal. I am now using fold-into-vec + r/map with zipmap which is better, > > but > > I wouldn't consider that optimal. > > The bottleneck with fold-into-map is that merge just doesn't scale up > well and you end up doing a significant amount of merging. > Currently a merge operation takes elements one at a time from one map, > and adds them one at a time to another. > > This is incredibly wasteful for large maps as you already have two > "pre-sorted" tries, and can pair each branch together recursively. If > each branch node stored the number of child nodes, then you can assign > different threads to work on different branches as well. This would be > perfect for reducers, but from a quick look it didn't appear that any > of the key internals were exposed to be taken advantage of. > > Unfortunately I haven't discovered a way to facilitate more efficient > merging without modifying the actual Clojure core source. I guess this > is my next step. Just stumbled over this old thread again and wondered if you found time to look into this again? -- Wolodja <babi...@gmail.com> 4096R/CAF14EFC 081C B7CD FF04 2BA9 94EA 36B2 8B7F 7D30 CAF1 4EFC
signature.asc
Description: Digital signature
