[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiri Daněk updated DISPATCH-337: Description: Valgrind shows huge memory leaks while running 2 interconnected routers with 2 parallel senders connected to the one router and 2 parallel receivers connected to the other router. The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: PROTON-1115 However, the rest of the leaks are from qdrouterd. was: Valgrind shows huge memory leaks while running 2 interconnected routers with 2 parallel senders connected to the one router and 2 parallel receivers connected to the other router. The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: https://issues.apache.org/jira/browse/PROTON-1115 However, the rest of the leaks are from qdrouterd. > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Major > Labels: memory-bug > Fix For: Backlog > > Attachments: LNP-1_Huge_memory.png, LNP-1_Leak_starts.png, > LNP-1_not_accepting_connections.png, Memory_usage_first_run_no_SSL.png, > Memory_usage_subsequent_run_no_SSL.png, Rapid_perm_memory_increase.png, > Subsequent_memory_increase.png, Tim-Router-3-huge-memory-usage.png, > Tim_Router_3.png, Tim_Routers_3_and_6_further_leaks.png, config1.conf, > config2.conf, val2_receiver.txt, val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Ross updated DISPATCH-337: -- Priority: Major (was: Critical) > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Major > Labels: memory-bug > Fix For: Backlog > > Attachments: LNP-1_Huge_memory.png, LNP-1_Leak_starts.png, > LNP-1_not_accepting_connections.png, Memory_usage_first_run_no_SSL.png, > Memory_usage_subsequent_run_no_SSL.png, Rapid_perm_memory_increase.png, > Subsequent_memory_increase.png, Tim-Router-3-huge-memory-usage.png, > Tim_Router_3.png, Tim_Routers_3_and_6_further_leaks.png, config1.conf, > config2.conf, val2_receiver.txt, val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > https://issues.apache.org/jira/browse/PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti updated DISPATCH-337: Labels: memory-bug (was: ) > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Critical > Labels: memory-bug > Fix For: Backlog > > Attachments: LNP-1_Huge_memory.png, LNP-1_Leak_starts.png, > LNP-1_not_accepting_connections.png, Memory_usage_first_run_no_SSL.png, > Memory_usage_subsequent_run_no_SSL.png, Rapid_perm_memory_increase.png, > Subsequent_memory_increase.png, Tim-Router-3-huge-memory-usage.png, > Tim_Router_3.png, Tim_Routers_3_and_6_further_leaks.png, config1.conf, > config2.conf, val2_receiver.txt, val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > https://issues.apache.org/jira/browse/PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Sharda updated DISPATCH-337: --- Attachment: LNP-1_not_accepting_connections.png LNP-1_Leak_starts.png LNP-1_Huge_memory.png One router on a machine (LNP-1) in a cluster of 5 completely connected routers suddenly started growing in memory and stopped accepting incoming connections. We cannot even run qdstat on that router to know its status. > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Critical > Attachments: LNP-1_Huge_memory.png, LNP-1_Leak_starts.png, > LNP-1_not_accepting_connections.png, Memory_usage_first_run_no_SSL.png, > Memory_usage_subsequent_run_no_SSL.png, Rapid_perm_memory_increase.png, > Subsequent_memory_increase.png, Tim-Router-3-huge-memory-usage.png, > Tim_Router_3.png, Tim_Routers_3_and_6_further_leaks.png, config1.conf, > config2.conf, val2_receiver.txt, val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > https://issues.apache.org/jira/browse/PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Sharda updated DISPATCH-337: --- Attachment: Tim-Router-3-huge-memory-usage.png Tim_Routers_3_and_6_further_leaks.png Tim_Router_3.png We are running a network of 5 routers (0.6.1 and Proton 0.14.0) on 5 bare metal machines having Ubuntu 16.04.1 LTS. These charts show that routers leak memory every time clients connect. There are no Undelivered/pending messages. > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Critical > Attachments: Memory_usage_first_run_no_SSL.png, > Memory_usage_subsequent_run_no_SSL.png, Rapid_perm_memory_increase.png, > Subsequent_memory_increase.png, Tim-Router-3-huge-memory-usage.png, > Tim_Router_3.png, Tim_Routers_3_and_6_further_leaks.png, config1.conf, > config2.conf, val2_receiver.txt, val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > https://issues.apache.org/jira/browse/PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Sharda updated DISPATCH-337: --- Attachment: Memory_usage_subsequent_run_no_SSL.png Memory_usage_first_run_no_SSL.png Attached two files: Memory_usage_first_run_no_SSL.png and Memory_usage_subsequent_run_no_SSL.png I ran another test - two routers connected to each other, several senders connecting to first router and 1 receiver connecting to the second router. All connections were insecure (no SSL at all). I see the same type of memory leaks but the pace of growth in resident memory usage is nearly halved. SSL amplifies the leaks but they occur without SSL also. > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Critical > Attachments: Memory_usage_first_run_no_SSL.png, > Memory_usage_subsequent_run_no_SSL.png, Rapid_perm_memory_increase.png, > Subsequent_memory_increase.png, config1.conf, config2.conf, > val2_receiver.txt, val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > https://issues.apache.org/jira/browse/PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Sharda updated DISPATCH-337: --- Attachment: Subsequent_memory_increase.png Rapid_perm_memory_increase.png Two files attached: Rapid_perm_memory_increase.png and Subsequent_memory_increase.png We ran two interconnected routers having 2-way SSL connection between them and accepting 2-way SSL connections from clients. We connected several senders to one router and several receivers to the other. Total of 250K messages were sent from one end to the other. We saw rapid increase in memory usage of both the routers. This memory never became low again and a subsequent identical run increased the memory further. Since the routers are never intended to be terminated, such memory leaks on all subsequent connections and data transfers will eventually lead to routers being killed. > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Critical > Attachments: Rapid_perm_memory_increase.png, > Subsequent_memory_increase.png, config1.conf, config2.conf, > val2_receiver.txt, val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > https://issues.apache.org/jira/browse/PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-337) Huge memory leaks in Qpid Dispatch router
[ https://issues.apache.org/jira/browse/DISPATCH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Sharda updated DISPATCH-337: --- Attachment: val2_sender.txt val2_receiver.txt config2.conf config1.conf Configuration files for two routers and the valgrind output with debug build of Qpid Dispatch running with 2 senders and 2 receivers. > Huge memory leaks in Qpid Dispatch router > - > > Key: DISPATCH-337 > URL: https://issues.apache.org/jira/browse/DISPATCH-337 > Project: Qpid Dispatch > Issue Type: Bug >Affects Versions: 0.6.0 > Environment: Debian 8.3, Apache Qpid Proton 0.12.2 for drivers and > dependencies, Hardware: 2 CPUs, 15 GB RAM, 60 GB HDD on 2 separate machines >Reporter: Vishal Sharda >Priority: Critical > Attachments: config1.conf, config2.conf, val2_receiver.txt, > val2_sender.txt > > > Valgrind shows huge memory leaks while running 2 interconnected routers with > 2 parallel senders connected to the one router and 2 parallel receivers > connected to the other router. > The CRYPTO leak that is coming from Qpid Proton 0.12.2 is already fixed here: > https://issues.apache.org/jira/browse/PROTON-1115 > However, the rest of the leaks are from qdrouterd. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
