Groovy Version: 2.1.6 JVM: 1.6.0_51 Vendor: Apple Inc. OS: Mac OS X Java(TM) SE Runtime Environment (build 1.6.0_51-b11-457-11M4509)
##### Heap utilization statistics [MB] ##### [1] Used Memory:1 Free Memory:79 Total Memory:81 Max Memory:123 [1] http://viralpatel.net/blogs/getting-jvm-heap-size-used-memory-total-memory-using-java-runtime/ I'm not a groovy expert... is there a way to query it about what it thinks its heap size is? Ben On Sun, Oct 13, 2013 at 8:35 AM, domi <[email protected]> wrote: > merged and started an other build: > https://ci.jenkins-ci.org/view/Infrastructure/job/infra_statistics/95/ > some other ideas/questions: > - groovy version? > - java version? > - memory (heap)? > regards Domi > > > On 11.10.2013, at 00:13, Benjamin Lau <[email protected]> wrote: > >> Some performance stats: >> With no indexes (didn't even finish running killed when it got to envfile): >> real 632m21.430s user 48m28.973s sys 84m20.053s >> >> + CREATE INDEX plugin_name on plugin (name); >> real 102m17.946s user 7m19.054s sys 15m54.729s >> >> + CREATE INDEX jenkins_version on jenkins (version); >> real 24m53.078s user 3m38.248s sys 3m38.441s >> >> + CREATE INDEX plugin_month on plugin (month); >> real 76m37.949s user 4m54.953s sys 10m2.820s >> Note: not exactly sure how adding an addition index winds up with the >> script running slower... >> >> + CREATE INDEX plugin_namemonth on plugin (name,month); >> real 1m13.944s user 0m44.397s sys 0m7.887s >> >> + DROP INDEX plugin_month; >> real 3m37.779s user 0m49.016s sys 0m28.856s >> >> So based on these results I've sent another pull request[1] which adds >> 3 additional indexes on top of the plugin.name one from before: >> CREATE INDEX jenkins_version on jenkins (version); >> CREATE INDEX plugin_month on plugin (month); >> CREATE INDEX plugin_namemonth on plugin (name,month); >> >> I ran collectNumbers.groovy after making the changes. Here's the >> performance results from that: >> mkdir -p target && time groovy collectNumbers.groovy >> ../jenkins-ci.org/census/20*.gz >> real 66m34.673s user 48m30.316s sys 19m25.083s >> >> time groovy createJson.groovy >> real 3m33.182s user 0m48.383s sys 0m24.900s >> >> Ben >> [1] https://github.com/jenkinsci/infra-statistics/pull/2 >> >> On Wed, Oct 9, 2013 at 4:05 PM, Benjamin Lau <[email protected]> >> wrote: >>> I'll see if I can run a benchmark of the old version of the code vs >>> the indexed version... I was getting the same runs forever behavior on >>> my computer before I added the indexing and could run the whole job in >>> a "reasonable"[1] amount of time after the change... I'm completely >>> stumped as to why this remained broken. >>> >>> Ben >>> >>> On Wed, Oct 9, 2013 at 3:55 PM, Kohsuke Kawaguchi >>> <[email protected]> wrote: >>>> On 10/08/2013 09:42 PM, Benjamin Lau wrote: >>>>> >>>>> It seems to at least be running... but the performance characteristics >>>>> are nothing like what I'm seeing on the machine I have at home. >>>>> >>>>> Do you know anything about what kind of hardware ci.jenkins-ci.org is? >>>> >>>> >>>> I doubt if the slow down of this degree can be explained by hardware, but >>>> here it is: >>>> >>>>> kohsuke@cucumber:~$ cat /proc/cpuinfo >>>>> processor : 0 >>>>> vendor_id : AuthenticAMD >>>>> cpu family : 16 >>>>> model : 2 >>>>> model name : AMD Athlon(tm) 7850 Dual-Core Processor >>>>> stepping : 3 >>>>> cpu MHz : 1400.000 >>>>> cache size : 512 KB >>>>> physical id : 0 >>>>> siblings : 2 >>>>> core id : 0 >>>>> cpu cores : 2 >>>>> apicid : 0 >>>>> initial apicid : 0 >>>>> fpu : yes >>>>> fpu_exception : yes >>>>> cpuid level : 5 >>>>> wp : yes >>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca >>>>> cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt >>>>> pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc >>>>> extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic >>>>> cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs >>>>> bogomips : 5599.89 >>>>> TLB size : 1024 4K pages >>>>> clflush size : 64 >>>>> cache_alignment : 64 >>>>> address sizes : 48 bits physical, 48 bits virtual >>>>> power management: ts ttp tm stc 100mhzsteps hwpstate >>>>> >>>>> processor : 1 >>>>> vendor_id : AuthenticAMD >>>>> cpu family : 16 >>>>> model : 2 >>>>> model name : AMD Athlon(tm) 7850 Dual-Core Processor >>>>> stepping : 3 >>>>> cpu MHz : 1400.000 >>>>> cache size : 512 KB >>>>> physical id : 0 >>>>> siblings : 2 >>>>> core id : 1 >>>>> cpu cores : 2 >>>>> apicid : 1 >>>>> initial apicid : 1 >>>>> fpu : yes >>>>> fpu_exception : yes >>>>> cpuid level : 5 >>>>> wp : yes >>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca >>>>> cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt >>>>> pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc >>>>> extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic >>>>> cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs >>>>> bogomips : 5600.08 >>>>> TLB size : 1024 4K pages >>>>> clflush size : 64 >>>>> cache_alignment : 64 >>>>> address sizes : 48 bits physical, 48 bits virtual >>>>> power management: ts ttp tm stc 100mhzsteps hwpstate >>>>> >>>>> kohsuke@cucumber:~$ cat /proc/meminfo >>>>> MemTotal: 8065552 kB >>>>> MemFree: 982308 kB >>>>> Buffers: 668732 kB >>>>> Cached: 3066860 kB >>>>> SwapCached: 39548 kB >>>>> Active: 4339220 kB >>>>> Inactive: 1716388 kB >>>>> Active(anon): 1818532 kB >>>>> Inactive(anon): 571804 kB >>>>> Active(file): 2520688 kB >>>>> Inactive(file): 1144584 kB >>>>> Unevictable: 0 kB >>>>> Mlocked: 0 kB >>>>> SwapTotal: 23631572 kB >>>>> SwapFree: 23049072 kB >>>>> Dirty: 1412 kB >>>>> Writeback: 0 kB >>>>> AnonPages: 2299860 kB >>>>> Mapped: 62588 kB >>>>> Shmem: 70320 kB >>>>> Slab: 754624 kB >>>>> SReclaimable: 700616 kB >>>>> SUnreclaim: 54008 kB >>>>> KernelStack: 3848 kB >>>>> PageTables: 48692 kB >>>>> NFS_Unstable: 0 kB >>>>> Bounce: 0 kB >>>>> WritebackTmp: 0 kB >>>>> CommitLimit: 27664348 kB >>>>> Committed_AS: 3852228 kB >>>>> VmallocTotal: 34359738367 kB >>>>> VmallocUsed: 374416 kB >>>>> VmallocChunk: 34359341476 kB >>>>> HardwareCorrupted: 0 kB >>>>> HugePages_Total: 0 >>>>> HugePages_Free: 0 >>>>> HugePages_Rsvd: 0 >>>>> HugePages_Surp: 0 >>>>> Hugepagesize: 2048 kB >>>>> DirectMap4k: 9792 kB >>>>> DirectMap2M: 1955840 kB >>>>> DirectMap1G: 6291456 kB >>>> >>>>> >>>>> >>>>> kohsuke@cucumber:~$ df -h >>>>> Filesystem Size Used Avail Use% Mounted on >>>>> /dev/sda1 895G 438G 412G 52% / >>>>> none 3.9G 208K 3.9G 1% /dev >>>>> none 3.9G 0 3.9G 0% /dev/shm >>>>> none 3.9G 67M 3.8G 2% /var/run >>>>> none 3.9G 0 3.9G 0% /var/lock >>>>> none 3.9G 0 3.9G 0% /lib/init/rw >>>> >>>> >>>> >>>> -- >>>> Kohsuke Kawaguchi | CloudBees, Inc. | http://cloudbees.com/ >>>> Try Jenkins Enterprise, our professional version of Jenkins >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Jenkins Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> For more options, visit https://groups.google.com/groups/opt_out. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
