Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[Alexander Yakushev]

Love to hear that!

On Tuesday, April 17, 2018 at 2:57:18 PM UTC+3, Kimmo Koskinen wrote:
>
> We just tried this library with a colleague while trying to track down a 
> memory leak when parsing a jdbc result set and saw a stunning victory :)
> With clj-memory-meter we were able to see that lazy parsing of json 
> strings left references to quite large buffers (in comparison to actual 
> data) to each item in the sequence, while strict parsing naturally removed 
> the problem.
>
> clj-memory-meter showed it's usefulness, so a big thanks for the library! 
> :)
>
> keskiviikko 7. maaliskuuta 2018 18.13.23 UTC+2 Alexander Yakushev 
> kirjoitti:
>>
>> Looks like it's because of JDK9. I created a Github issue, let's take it 
>> there: https://github.com/clojure-goes-fast/clj-memory-meter/issues/1
>>
>> On Wednesday, March 7, 2018 at 5:49:07 PM UTC+2, 路Ricardo M. wrote:
>>>
>>> Java Version:
>>> java version "9.0.4"
>>> Java(TM) SE Runtime Environment (build 9.0.4+11)
>>> Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
>>> System: Arch Linux
>>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[Kimmo Koskinen]

We just tried this library with a colleague while trying to track down a 
memory leak when parsing a jdbc result set and saw a stunning victory :)
With clj-memory-meter we were able to see that lazy parsing of json strings 
left references to quite large buffers (in comparison to actual data) to 
each item in the sequence, while strict parsing naturally removed the 
problem.

clj-memory-meter showed it's usefulness, so a big thanks for the library! :)

keskiviikko 7. maaliskuuta 2018 18.13.23 UTC+2 Alexander Yakushev kirjoitti:
>
> Looks like it's because of JDK9. I created a Github issue, let's take it 
> there: https://github.com/clojure-goes-fast/clj-memory-meter/issues/1
>
> On Wednesday, March 7, 2018 at 5:49:07 PM UTC+2, 路Ricardo M. wrote:
>>
>> Java Version:
>> java version "9.0.4"
>> Java(TM) SE Runtime Environment (build 9.0.4+11)
>> Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
>> System: Arch Linux
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[Alexander Yakushev]

Looks like it's because of JDK9. I created a Github issue, let's take it 
there: https://github.com/clojure-goes-fast/clj-memory-meter/issues/1

On Wednesday, March 7, 2018 at 5:49:07 PM UTC+2, 路Ricardo M. wrote:
>
> Java Version:
> java version "9.0.4"
> Java(TM) SE Runtime Environment (build 9.0.4+11)
> Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
> System: Arch Linux
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[numbch...@gmail.com]

Java Version:
java version "9.0.4"
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
System: Arch Linux




user=> (require '[clj-memory-meter.core :as mm])

CompilerException java.lang.reflect.InvocationTargetException,
compiling:(clj_memory_meter/core.clj:50:1)
user=> *e
#error {
 :cause "Can not attach to current VM"
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "java.lang.reflect.InvocationTargetException,
compiling:(clj_memory_meter/core.clj:50:1)"
   :at [clojure.lang.Compiler load "Compiler.java" 7391]}
  {:type java.lang.reflect.InvocationTargetException
   :message nil
   :at [jdk.internal.reflect.NativeMethodAccessorImpl invoke0
"NativeMethodAccessorImpl.java" -2]}
  {:type java.io.IOException
   :message "Can not attach to current VM"
   :at [sun.tools.attach.HotSpotVirtualMachine 
"HotSpotVirtualMachine.java" 75]}]
 :trace
 [[sun.tools.attach.HotSpotVirtualMachine 
"HotSpotVirtualMachine.java" 75]
  [sun.tools.attach.VirtualMachineImpl  "VirtualMachineImpl.java" 56]
  [sun.tools.attach.AttachProviderImpl attachVirtualMachine
"AttachProviderImpl.java" 58]
  [com.sun.tools.attach.VirtualMachine attach "VirtualMachine.java" 207]
  [jdk.internal.reflect.NativeMethodAccessorImpl invoke0
"NativeMethodAccessorImpl.java" -2]
  [jdk.internal.reflect.NativeMethodAccessorImpl invoke
"NativeMethodAccessorImpl.java" 62]
  [jdk.internal.reflect.DelegatingMethodAccessorImpl invoke
"DelegatingMethodAccessorImpl.java" 43]
  [java.lang.reflect.Method invoke "Method.java" 564]
  [clj_memory_meter.core$mk_vm invokeStatic "core.clj" 42]
  [clj_memory_meter.core$mk_vm invoke "core.clj" 38]
  [clj_memory_meter.core$load_jamm_agent invokeStatic "core.clj" 45]
  [clj_memory_meter.core$load_jamm_agent invoke "core.clj" 44]
  [clj_memory_meter.core$eval1822 invokeStatic "core.clj" 50]
  [clj_memory_meter.core$eval1822 invoke "core.clj" 50]
  [clojure.lang.Compiler eval "Compiler.java" 6927]
  [clojure.lang.Compiler load "Compiler.java" 7379]

  [clojure.lang.RT loadResourceScript "RT.java" 372]
  [clojure.lang.RT loadResourceScript "RT.java" 363]
  [clojure.lang.RT load "RT.java" 453]
  [clojure.lang.RT load "RT.java" 419]
  [clojure.core$load$fn__5677 invoke "core.clj" 5893]
  [clojure.core$load invokeStatic "core.clj" 5892]
  [clojure.core$load doInvoke "core.clj" 5876]
  [clojure.lang.RestFn invoke "RestFn.java" 408]
  [clojure.core$load_one invokeStatic "core.clj" 5697]
  [clojure.core$load_one invoke "core.clj" 5692]
  [clojure.core$load_lib$fn__5626 invoke "core.clj" 5737]
  [clojure.core$load_lib invokeStatic "core.clj" 5736]
  [clojure.core$load_lib doInvoke "core.clj" 5717]
  [clojure.lang.RestFn applyTo "RestFn.java" 142]
  [clojure.core$apply invokeStatic "core.clj" 648]
  [clojure.core$load_libs invokeStatic "core.clj" 5774]
  [clojure.core$load_libs doInvoke "core.clj" 5758]
  [clojure.lang.RestFn applyTo "RestFn.java" 137]
  [clojure.core$apply invokeStatic "core.clj" 648]
  [clojure.core$require invokeStatic "core.clj" 5796]
  [clojure.core$require doInvoke "core.clj" 5796]
  [clojure.lang.RestFn invoke "RestFn.java" 408]
  [user$eval1800 invokeStatic nil 1]
  [user$eval1800 invoke nil 1]
  [clojure.lang.Compiler eval "Compiler.java" 6927]
  [clojure.lang.Compiler eval "Compiler.java" 6890]
  [clojure.core$eval invokeStatic "core.clj" 3105]
  [clojure.core$eval invoke "core.clj" 3101]
  [clojure.main$repl$read_eval_print__7408$fn__7411 invoke "main.clj" 240]
  [clojure.main$repl$read_eval_print__7408 invoke "main.clj" 240]
  [clojure.main$repl$fn__7417 invoke "main.clj" 258]
  [clojure.main$repl invokeStatic "main.clj" 258]
  [clojure.main$repl doInvoke "main.clj" 174]
  [clojure.lang.RestFn invoke "RestFn.java" 1523]
  [clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__5508
invoke "interruptible_eval.clj" 87]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 646]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1881]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1881]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [clojure.tools.nrepl.middleware.interruptible_eval$evaluate invokeStatic
"interruptible_eval.clj" 85]
  [clojure.tools.nrepl.middleware.interruptible_eval$evaluate invoke
"interruptible_eval.clj" 55]

[clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__5553$fn__5556
invoke "interruptible_eval.clj" 224]
  [clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__5548
invoke "interruptible_eval.clj" 192]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.util.concurrent.ThreadPoolExecutor runWorker
"ThreadPoolExecutor.java" 1167]
  [java.util.concurrent.ThreadPoolExecutor$Worker run
"ThreadPoolExecutor.java" 641]
  [java.lang.Thread run "Thread.java" 844]]}
user=>




[stardiviner] GPG key ID: 47C32433
IRC(freeenode): stardiviner 

Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[Juraj Martinka]

Works great for me. Thank you very much for publishing this!
Previously, I tried to use jol-cli  
but it was much more 
cumbersome.

On Monday, 5 March 2018 11:56:20 UTC+1, Alexander Yakushev wrote:
>
> I'm happy to release the first version of clj-memory-meter 
> . It's a thin 
> wrapper around jamm  which allows 
> measuring how much space an arbitrary object occupies. clj-memory-meter is 
> usable from the REPL, can be loaded on-demand, and doesn't require to 
> launch JVM with special parameters.
>
> Blog post with a little more detail and usage examples: 
> http://clojure-goes-fast.com/blog/introspection-tool-object-memory-meter/
>
> To start, using add [com.clojure-goes-fast/clj-memory-meter "0.1.0"], or 
> run the following with clj:
>
> $ clj -Sdeps "{:deps {com.clojure-goes-fast/clj-memory-meter 
> {:mvn/version \"0.1.0\"}}}"
> Clojure 1.9.0
>
> ;; Nevermind the warning, it doesn't break the library.
> user=> (require '[clj-memory-meter.core :as mm])
> objc[59881]: Class JavaLaunchHelper is implemented in both 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java 
> (0x1089814c0) and 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/jre/lib/libinstrument.dylib
>  
> (0x10d9dc4e0). One of the two will be used. Which one is undefined.
> nil
>
> user=> (mm/measure (vec (range 10)))
> "2.8 MB"
>
> user=> (mm/measure (list 1 2 3 4 5) :debug true)
>
> root [clojure.lang.PersistentList] 320 bytes (40 bytes)
>   |
>   +--_first [java.lang.Long] 24 bytes (24 bytes)
>   |
>   +--_rest [clojure.lang.PersistentList] 256 bytes (40 bytes)
> |
> +--_first [java.lang.Long] 24 bytes (24 bytes)
> |
> +--_rest [clojure.lang.PersistentList] 192 bytes (40 bytes)
>   |
>   +--_first [java.lang.Long] 24 bytes (24 bytes)
>   |
>   +--_rest [clojure.lang.PersistentList] 128 bytes (40 bytes)
> |
> +--_first [java.lang.Long] 24 bytes (24 bytes)
> |
> +--_rest [clojure.lang.PersistentList] 64 bytes (40 bytes)
>   |
>   +--_first [java.lang.Long] 24 bytes (24 bytes)
>
> "320 B"
>
> I hope this library will be useful for you. Enjoy!
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[Sean Corfield]

Could you try that again and immediately after getting the exception from the 
require, type in:

  *e

(instead of trying the mm/measure call)

In the exception that prints out, look for the Caused by portion of the 
exception. That’s going to be the underlying exception that will help us help 
you debug this.

Also, what version of Java are you using? (java -version)

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com <clojure@googlegroups.com> on behalf of 
numbch...@gmail.com <numbch...@gmail.com>
Sent: Tuesday, March 6, 2018 8:38:34 PM
To: clojure@googlegroups.com
Subject: Re: [ANN] clj-memory meter – measure the memory used by arbitrary 
objects

Got error when requiring:

clj -Sdeps "{:deps {com.clojure-goes-fast/clj-memory-meter {:mvn/version 
\"0.1.0\"}}}"
Clojure 1.9.0
user=> (require '[clj-memory-meter.core :as mm])
CompilerException java.lang.reflect.InvocationTargetException, 
compiling:(clj_memory_meter/core.clj:50:1)
user=> (mm/measure (vec (range 10)))
CompilerException java.lang.RuntimeException: No such namespace: mm, 
compiling:(NO_SOURCE_PATH:2:1)
user=>


[stardiviner] GPG key ID: 47C32433
IRC(freeenode): stardiviner Twitter:  @numbchild
Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
Blog: http://stardiviner.github.io/

On Tue, Mar 6, 2018 at 7:44 PM, Alexander Yakushev 
<unlo...@bytopia.org<mailto:unlo...@bytopia.org>> wrote:
user=> (count nn)
25000

user=> (mm/measure nn)
"940.3 KB"

;; Still shared though!
user=>(mm/measure [mm nn])
"1.8 MB"

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to 
clojure@googlegroups.com<mailto:clojure@googlegroups.com>
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com<mailto:clojure%2bunsubscr...@googlegroups.com>
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
clojure+unsubscr...@googlegroups.com<mailto:clojure+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
clojure+unsubscr...@googlegroups.com<mailto:clojure+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[numbch...@gmail.com]

Got error when requiring:

clj -Sdeps "{:deps {com.clojure-goes-fast/clj-memory-meter {:mvn/version
\"0.1.0\"}}}"
Clojure 1.9.0
user=> (require '[clj-memory-meter.core :as mm])
CompilerException java.lang.reflect.InvocationTargetException,
compiling:(clj_memory_meter/core.clj:50:1)
user=> (mm/measure (vec (range 10)))
CompilerException java.lang.RuntimeException: No such namespace: mm,
compiling:(NO_SOURCE_PATH:2:1)
user=>


[stardiviner] GPG key ID: 47C32433
IRC(freeenode): stardiviner Twitter:  @numbchild
Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
Blog: http://stardiviner.github.io/

On Tue, Mar 6, 2018 at 7:44 PM, Alexander Yakushev 
wrote:

> user=> (count nn)
> 25000
>
> user=> (mm/measure nn)
> "940.3 KB"
>
> ;; Still shared though!
> user=>(mm/measure [mm nn])
> "1.8 MB"
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[Alexander Yakushev]

user=> (count nn)
25000

user=> (mm/measure nn)
"940.3 KB"

;; Still shared though!
user=>(mm/measure [mm nn])
"1.8 MB"

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-memory meter – measure the memory used by arbitrary objects

[Matching Socks]

Unrealized lazy sequence...

user=> (def mm (filter even? (range 10)))
#'user/mm
user=> (mm/measure mm)
"160 B"


Realized...

user=> (count mm)
5
user=> (mm/measure mm)
"1.8 MB"


Structural sharing?

user=> (def nn (drop 25000 mm))
#'user/nn
user=> (mm/measure nn)
"1.8 MB"
user=> (mm/measure [mm nn])
"1.8 MB"


Can we free some of that memory?

user=> (def mm nil)
#'user/mm
user=> (mm/measure nn)
"1.8 MB"
user=> (System/gc)
nil
user=> (mm/measure nn)
"1.8 MB"


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.