Hi all,
I'm trying to investigate strange memory consumption, but
I have no explanation
.
Tested data is 100kb json text.
Tested script:
var fs = require('fs');
var data = fs.readFileSync(__dirname + '/' +'chartype.letter.json', 'utf8');
var p0 = JSON.parse(data);
var p1 = JSON.parse(data);
var p2 = JSON.parse(data);
var p3 = JSON.parse(data);
var p4 = JSON.parse(data);
var p5 = JSON.parse(data);
var p6 = JSON.parse(data);
var p7 = JSON.parse(data);
var p8 = JSON.parse(data);
var p9 = JSON.parse(data);
Empty case:
/usr/bin/time -f "%MKb" ./node empty.js
35360Kb
So expected result for main case will be empty case + data * count_times
and it should be something like 35360Kb + 100Kb * 10 ~~ 37Mb
Main case:
/usr/bin/time -f "%MKb" ./node bench.js
276672Kb
Actual result is 270Mb.
Running mtrace <https://github.com/Jimbly/node-mtrac> :
Addr Size Count Traffic
Module Symbol Offs
-------------- --------- ----- -------
-------------------------------- ------------------------------ ------
0x6b3bb7 0 0 2
./build/bin/node ares_init_options +0x857
0x6b3df7 0 0 2
./build/bin/node ares_init_options +0xa97
0x6b93c6 0 0 2
./build/bin/node ares__read_line +0xd6
0x7ff99d96aebb 0 0 6
/lib/x86_64-linux-gnu/libc.so.6
0x7ff99d984ab2 3 1 3
/lib/x86_64-linux-gnu/libc.so.6 __strdup +0x22
0x6b41e0 360 1 5
./build/bin/node ares_init_options +0xe80
0x90a519 928 4 738
./build/bin/node _ZN2v88internal8Malloced3NewEm +0x9
0x7ff99e44936d 5270 8 870
/usr/lib/x86_64-linux-gnu/lib... _Znwm +0x1d
0x6b33da 74216 1 1
./build/bin/node ares_init_options +0x7a
-------------- --------- ----- -------
-------------------------------- ------------------------------ ------
Totals 80777 15 1629
Traising GC.
./build/bin/node --trace_gc --trace_gc_verbose ../bench.js
[26109] 19 ms: Scavenge 2.1 (34.0) -> 2.0 (36.0) MB, 0 ms [allocation
failure].
[26109] Memory allocator, used: 36880 KB, available: 1462256 KB
[26109] New space, used: 251 KB, available: 772 KB,
committed: 2048 KB
[26109] Old pointers, used: 708 KB, available: 1 KB,
committed: 1519 KB
[26109] Old data space, used: 570 KB, available: 0 KB,
committed: 1199 KB
[26109] Code space, used: 395 KB, available: 0 KB,
committed: 996 KB
[26109] Map space, used: 83 KB, available: 0 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 0 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1461215 KB,
committed: 0 KB
[26109] All spaces, used: 2024 KB, available: 773 KB,
committed: 6019 KB
[26109] Total time spent in GC : 0 ms
[26109] 20 ms: Scavenge 2.7 (36.0) -> 2.7 (37.0) MB, 0 ms [allocation
failure].
[26109] Memory allocator, used: 37904 KB, available: 1461232 KB
[26109] New space, used: 251 KB, available: 772 KB,
committed: 2048 KB
[26109] Old pointers, used: 716 KB, available: 1 KB,
committed: 1519 KB
[26109] Old data space, used: 1316 KB, available: 0 KB,
committed: 2207 KB
[26109] Code space, used: 395 KB, available: 0 KB,
committed: 996 KB
[26109] Map space, used: 83 KB, available: 0 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 0 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1460191 KB,
committed: 0 KB
[26109] All spaces, used: 2778 KB, available: 773 KB,
committed: 7027 KB
[26109] Total time spent in GC : 0 ms
[26109] 20 ms: Scavenge 3.6 (37.0) -> 3.6 (37.0) MB, 0 ms [allocation
failure].
[26109] Memory allocator, used: 37904 KB, available: 1461232 KB
[26109] New space, used: 503 KB, available: 1544 KB,
committed: 4096 KB
[26109] Old pointers, used: 908 KB, available: 1 KB,
committed: 1519 KB
[26109] Old data space, used: 1818 KB, available: 0 KB,
committed: 2207 KB
[26109] Code space, used: 395 KB, available: 0 KB,
committed: 996 KB
[26109] Map space, used: 83 KB, available: 0 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 0 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1460191 KB,
committed: 0 KB
[26109] All spaces, used: 3724 KB, available: 1545 KB,
committed: 9075 KB
[26109] Total time spent in GC : 0 ms
[26109] 22 ms: Scavenge 5.1 (37.0) -> 5.1 (39.0) MB, 0 ms [allocation
failure].
[26109] Memory allocator, used: 39952 KB, available: 1459184 KB
[26109] New space, used: 503 KB, available: 1544 KB,
committed: 4096 KB
[26109] Old pointers, used: 908 KB, available: 1 KB,
committed: 1519 KB
[26109] Old data space, used: 3326 KB, available: 2 KB,
committed: 4223 KB
[26109] Code space, used: 395 KB, available: 0 KB,
committed: 996 KB
[26109] Map space, used: 83 KB, available: 0 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 0 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1458143 KB,
committed: 0 KB
[26109] All spaces, used: 5233 KB, available: 1547 KB,
committed: 11090 KB
[26109] Total time spent in GC : 0 ms
[26109] 24 ms: Scavenge 6.6 (39.0) -> 6.6 (40.0) MB, 1 ms [allocation
failure].
[26109] Memory allocator, used: 40976 KB, available: 1458160 KB
[26109] New space, used: 1007 KB, available: 3088 KB,
committed: 8192 KB
[26109] Old pointers, used: 908 KB, available: 1 KB,
committed: 1519 KB
[26109] Old data space, used: 4331 KB, available: 2 KB,
committed: 5230 KB
[26109] Code space, used: 395 KB, available: 0 KB,
committed: 996 KB
[26109] Map space, used: 83 KB, available: 0 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 0 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1457119 KB,
committed: 0 KB
[26109] All spaces, used: 6741 KB, available: 3091 KB,
committed: 16194 KB
[26109] Total time spent in GC : 1 ms
[26109] 28 ms: Mark-sweep 10.3 (41.0) -> 2.9 (41.0) MB, 2 ms
[allocation failure] [promotion limit reached].
[26109] Memory allocator, used: 42000 KB, available: 1457136 KB
[26109] New space, used: 0 KB, available: 4096 KB,
committed: 8192 KB
[26109] Old pointers, used: 1441 KB, available: 894 KB,
committed: 2527 KB
[26109] Old data space, used: 1092 KB, available: 3564 KB,
committed: 5230 KB
[26109] Code space, used: 384 KB, available: 612 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 62 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1456095 KB,
committed: 0 KB
[26109] All spaces, used: 2997 KB, available: 9343 KB,
committed: 17202 KB
[26109] Total time spent in GC : 3 ms
[26109] 30 ms: Scavenge 7.0 (41.0) -> 7.0 (44.0) MB, 1 ms [allocation
failure].
[26109] Memory allocator, used: 45072 KB, available: 1454064 KB
[26109] New space, used: 1007 KB, available: 3088 KB,
committed: 8192 KB
[26109] Old pointers, used: 1633 KB, available: 284 KB,
committed: 2527 KB
[26109] Old data space, used: 4109 KB, available: 3566 KB,
committed: 8254 KB
[26109] Code space, used: 384 KB, available: 612 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 62 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1453023 KB,
committed: 0 KB
[26109] All spaces, used: 7213 KB, available: 7726 KB,
committed: 20225 KB
[26109] Total time spent in GC : 4 ms
[26109] 32 ms: Scavenge 10.7 (45.0) -> 7.0 (45.0) MB, 0 ms
[allocation failure].
[26109] Memory allocator, used: 46096 KB, available: 1453040 KB
[26109] New space, used: 118 KB, available: 8073 KB,
committed: 16384 KB
[26109] Old pointers, used: 2401 KB, available: 894 KB,
committed: 3535 KB
[26109] Old data space, used: 4139 KB, available: 3566 KB,
committed: 8254 KB
[26109] Code space, used: 384 KB, available: 612 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 62 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1451999 KB,
committed: 0 KB
[26109] All spaces, used: 7123 KB, available: 13321 KB,
committed: 29425 KB
[26109] Total time spent in GC : 4 ms
[26109] 37 ms: Scavenge 14.7 (45.0) -> 14.5 (51.0) MB, 3 ms
[allocation failure].
[26109] Memory allocator, used: 52240 KB, available: 1446896 KB
[26109] New space, used: 2015 KB, available: 6176 KB,
committed: 16384 KB
[26109] Old pointers, used: 3169 KB, available: 1134 KB,
committed: 4543 KB
[26109] Old data space, used: 9150 KB, available: 3568 KB,
committed: 13292 KB
[26109] Code space, used: 384 KB, available: 612 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 62 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1445855 KB,
committed: 0 KB
[26109] All spaces, used: 14799 KB, available: 11667 KB,
committed: 35471 KB
[26109] Total time spent in GC : 7 ms
[26109] Increasing marking speed to 3 due to high promotion rate
[26109] 45 ms: Mark-sweep 20.4 (51.0) -> 16.3 (54.0) MB, 4 ms (+ 2 ms
in 35 steps since start of marking, biggest step 0.533936 ms) [allocation
failure] [promotion limit reached].
[26109] Memory allocator, used: 55312 KB, available: 1443824 KB
[26109] New space, used: 0 KB, available: 8192 KB,
committed: 16384 KB
[26109] Old pointers, used: 3117 KB, available: 1378 KB,
committed: 4543 KB
[26109] Old data space, used: 13075 KB, available: 3239 KB,
committed: 16316 KB
[26109] Code space, used: 384 KB, available: 612 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 62 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1442783 KB,
committed: 0 KB
[26109] All spaces, used: 16656 KB, available: 13597 KB,
committed: 38495 KB
[26109] Total time spent in GC : 11 ms
[26109] 50 ms: Scavenge 24.9 (54.0) -> 22.7 (56.0) MB, 2 ms
[allocation failure].
[26109] Memory allocator, used: 57360 KB, available: 1441776 KB
[26109] New space, used: 4030 KB, available: 12353 KB,
committed: 32768 KB
[26109] Old pointers, used: 4077 KB, available: 418 KB,
committed: 4543 KB
[26109] Old data space, used: 14701 KB, available: 3240 KB,
committed: 18331 KB
[26109] Code space, used: 384 KB, available: 612 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 61 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1440735 KB,
committed: 0 KB
[26109] All spaces, used: 23274 KB, available: 16798 KB,
committed: 56894 KB
[26109] Total time spent in GC : 13 ms
[26109] 65 ms: Mark-sweep 33.8 (57.0) -> 12.8 (58.0) MB, 5 ms (+ 3 ms
in 40 steps since start of marking, biggest step 0.824951 ms) [StackGuard
GC request] [GC in old space requested].
[26109] Memory allocator, used: 59408 KB, available: 1439728 KB
[26109] New space, used: 0 KB, available: 16384 KB,
committed: 32768 KB
[26109] Old pointers, used: 5407 KB, available: 911 KB,
committed: 6558 KB
[26109] Old data space, used: 7214 KB, available: 11116 KB,
committed: 18331 KB
[26109] Code space, used: 373 KB, available: 622 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 62 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1438687 KB,
committed: 0 KB
[26109] All spaces, used: 13075 KB, available: 29210 KB,
committed: 58910 KB
[26109] Total time spent in GC : 18 ms
[26109] 72 ms: Scavenge 28.5 (58.0) -> 20.3 (62.0) MB, 3 ms
[allocation failure].
[26109] Memory allocator, used: 63504 KB, available: 1435632 KB
[26109] New space, used: 4030 KB, available: 12353 KB,
committed: 32768 KB
[26109] Old pointers, used: 6943 KB, available: 1391 KB,
committed: 8574 KB
[26109] Old data space, used: 9356 KB, available: 10989 KB,
committed: 20347 KB
[26109] Code space, used: 373 KB, available: 622 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 61 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1434591 KB,
committed: 0 KB
[26109] All spaces, used: 20784 KB, available: 25531 KB,
committed: 62941 KB
[26109] Total time spent in GC : 21 ms
[26109] 76 ms: Scavenge 32.0 (62.0) -> 19.5 (62.0) MB, 1 ms
[allocation failure].
[26109] Memory allocator, used: 63504 KB, available: 1435632 KB
[26109] New space, used: 3053 KB, available: 13330 KB,
committed: 32768 KB
[26109] Old pointers, used: 6943 KB, available: 1391 KB,
committed: 8574 KB
[26109] Old data space, used: 9479 KB, available: 10866 KB,
committed: 20347 KB
[26109] Code space, used: 373 KB, available: 622 KB,
committed: 996 KB
[26109] Map space, used: 65 KB, available: 61 KB,
committed: 128 KB
[26109] Cell space, used: 14 KB, available: 113 KB,
committed: 128 KB
[26109] Large object space, used: 0 KB, available: 1434591 KB,
committed: 0 KB
[26109] All spaces, used: 19930 KB, available: 26385 KB,
committed: 62941 KB
[26109] Total time spent in GC : 22 ms
Why memory consumption is so HUGE? Why is so different?
--
Job board: http://jobs.nodejs.org/
New group rules:
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/nodejs/a96af07b-db64-4cb9-bf17-06c17aed173a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
