Thanks for the input Jozef. I just want to clarify that I am not doing any nested reduce calls within my code.
Alex On Monday, November 17, 2014 1:00:43 PM UTC+2, Jozef Wagner wrote: > > I would look for unncessary object creation inside reduction function (use > transients) and lazy seq realization (realize only what you really need). > Calling reduce inside reduce (inside reduce...) may easily make the > complexity of your code exponential. Consider caching of intermediate > results if that's the case. > > You can also consider switching to reducers/transducers, which may clean > your design. In that case, the call to reduce should be as high in > hierarchy as possible, ideally in your top level function, with other > functions dealing with the transformation of 'step' values. > > Jozef > > On Mon, Nov 17, 2014 at 11:28 AM, Alexander L. <alexand...@gmail.com > <javascript:>> wrote: > >> Hi all, >> >> I understand that the following question is a long shot without any >> proper proof/tests from my side but it's a little bit difficult to make a >> test case from the specific part of my app so I will just ask anyway in >> case anyone knows anything. >> >> The situation is like this: >> >> - I have a hashmap with *3386* items that I pass through few >> functions in order to append new keys or update existing ones to each >> hashmap entry. >> - Each hashmap item has 20 keys with various data types (mostly >> strings) >> - All my transformation functions use `reduce`. >> >> The problem: >> >> I have a top level function which I inside it I call 7 other functions >> (all written by me) and for some reason I haven't discovered is that it >> needs around 2 seconds to return a result even though the items aren't many. >> Now, i used `time` to benchmark each function and when I found which one >> is taking a lot of time to return, after I removed it, I discovered that >> the problem still existed but now moved on to a different function. >> >> I did a bunch of tests with those 3386 on the REPL and reduce but I >> didn't notice anything weird/slow so it must be a combination of things. >> Also, i doubt that this is a RAM problem, i have allocated 4GB for the JVM. >> >> So, my question is, has anyone every seen a situation like this with a >> bunch of `reduce` calls? Is there anything at all that I should check and >> maybe missed it? >> >> Regards >> >> Alex >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> <javascript:> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
