Re: what does future do after fn finish ?

2018-02-03 Thread Jacek Grzebyta 
HI,

So things works well. In terms of memory usage anyway. I use 50 main
threads (in process-data-by-condition-set) and 20 for replications
(process-growths). However I found that after good performance in the
beginning it slows down rapid and quick. VisualVM shows the threads usage
is poor - mainly they are waiting. Only 1 claypoole thread works + some
lower level threads managed probably by Clojure, etc. CPU usage is poor as
well. I guess the reason is on the lower - Java classes level with IO
operations bottleneck. IMO they are working by blocking. That why I wanted
to separate them. However IO speed operations go down as well.
I've tried async channels but that they were flooded and some data where
lost for some reason.

Regards,
Jacek


On 3 February 2018 at 00:14, Justin Smith  wrote:

> -> is just a list transform performed after reading your code into a list
> data structure containing symbols, and before compiling to byte code - it
> doesn't do anything directly.
>
> On Fri, Feb 2, 2018 at 3:55 PM Jacek Grzebyta 
> wrote:
>
>> OK I found what makes the memory leak.
>>
>> In the project I work with I use a  java Model class which is java
>> Collection proxy/facade for a single transaction. Unfortunately it's not
>> thread safe. In a few places I passed single instance of model into several
>> threads Also I requested the instance with -> which makes new thread as
>> well. I was surprised that -> makes trouble but after thinking that might
>> be expected. Especially that internally the wrapper doesn't do simple
>> mapping - it uses some iterator, etc. Anyway the machinery is fragile. It
>> seems I need to rewrite code and replace all multithreading parts by
>> something simpler.
>>
>>
>>
>> If you want see the stacktrace just look at:
>> https://github.com/jgrzebyta/triple-loader/issues/53
>>
>> On 2 February 2018 at 11:16, Jacek Grzebyta 
>> wrote:
>>
>>>
>>> On 2 February 2018 at 08:34, Niels van Klaveren <
>>> niels.vanklave...@gmail.com> wrote:
>>>
 +1 for Claypoole, it removed the needs of using agents or futures in
 95% of the cases in my code.


>>> Thanks a lot. I modify the code using claypoole. I imagine with-shutdown
>>> will close the pool properly after finish all tasks so there is no need to
>>> watch them?
>>>
>>> J
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] tools.deps.alpha 0.5.351, clojure tools 1.9.0.326

2018-02-03 Thread Alex Miller 
New releases of tools.deps.alpha and the clojure tools (clj) are now 
available. This release primarily enhances alias support. There are 3 new 
kinds of aliases that can be supplied with clj:

-O - for java option aliases. Example in deps.edn: {:aliases {:mem 
{:jvm-opts ["-Xms100m" "-Xmx500m"]}}}. If multiple :jvm-opts are specfiied 
with aliases, they concatenate. If command-line JVM options are specified 
with -J (an already existing feature), those are supplied last.

-M - for clojure.main option aliases. Example in deps.edn: {:aliases 
{:myapp {:main-opts ["com.foo.myapp" "--start-server"]}}}. If multiple 
:main-opts are specified with aliases, only the last one is used (they 
replace, not combine). If command-line main options are specified, they are 
added to the end of the last :main-opts specified.

-A - for any kind of alias. This can be used instead of -R,-C,-O, or -M and 
you may combine multiple kinds of alias keys in a single alias. For example 
a deps.edn like this:

{:aliases
 {:cljs {:extra-deps {org.clojure/clojurescript {:mvn/version "1.9.946"}}
 :main-opts ["-m" "cljs.repl.node"]}}}

Can then invoked with just "clj -A:cljs" to start a cljs repl.

One important caveat on -O and -M options... Supplying arguments that 
contain spaces and would require quoting on the command line do not yet 
work. You are most likely to encounter these if you try to use -e (for a 
Clojure expression) in a :main-opts or start a socket repl (which takes an 
edn string) in a :jvm-opts.

Some other minor bugs were fixed as well.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.