Public bug reported: Hello! 👋
We have Ubuntu OS-based servers running in both AWS and Azure clouds. These servers are handling thousands of connections, and we've been experiencing issues with TCP memory usage since upgrading to Ubuntu 22.04.3 from 22.04.2. $ cat /proc/net/sockstat sockets: used 6642 TCP: inuse 5962 orphan 0 tw 292 alloc 6008 mem 128989 UDP: inuse 5 mem 0 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0 As shown in the output below, even after stopping all possible services and closing all open connections, the TCP memory usage remains high and only decreases very slowly. $ cat /proc/net/sockstat sockets: used 138 TCP: inuse 2 orphan 0 tw 0 alloc 3 mem 128320 UDP: inuse 3 mem 0 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0 I have attached a screenshot of linear TCP memory usage growth, which we believe may indicate a TCP memory leak We've never had these issues before, and the only solution we've found so far is to reboot the node. Do you have any suggestions on how to troubleshoot further? Thank you for any help or guidance you can provide! ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-6.2.0-1015-aws 6.2.0-1015.15~22.04.1 ProcVersionSignature: Ubuntu 6.2.0-1015.15~22.04.1-aws 6.2.16 Uname: Linux 6.2.0-1015-aws aarch64 ApportVersion: 2.20.11-0ubuntu82.5 Architecture: arm64 CasperMD5CheckResult: unknown CloudArchitecture: aarch64 CloudID: aws CloudName: aws CloudPlatform: ec2 CloudRegion: us-west-2 CloudSubPlatform: metadata (http://169.254.169.254) Date: Mon Dec 4 13:13:08 2023 Ec2AMI: ami-095a68e28e781dfe1 Ec2AMIManifest: (unknown) Ec2Architecture: arm64 Ec2AvailabilityZone: us-west-2b Ec2Imageid: ami-095a68e28e781dfe1 Ec2InstanceType: m7g.large Ec2Instancetype: m7g.large Ec2Kernel: unavailable Ec2Ramdisk: unavailable Ec2Region: us-west-2 ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash RebootRequiredPkgs: Error: path contained symlinks. SourcePackage: linux-signed-aws-6.2 UpgradeStatus: No upgrade log present (probably fresh install) ** Affects: linux-signed-aws-6.2 (Ubuntu) Importance: Undecided Status: New ** Tags: apport-bug arm64 ec2-images jammy kernel-net ** Attachment added: "Screenshot 2023-12-04 at 14.27.09.png" https://bugs.launchpad.net/bugs/2045560/+attachment/5726215/+files/Screenshot%202023-12-04%20at%2014.27.09.png -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-signed-aws-6.2 in Ubuntu. https://bugs.launchpad.net/bugs/2045560 Title: Possible TCP memory leak Status in linux-signed-aws-6.2 package in Ubuntu: New Bug description: Hello! 👋 We have Ubuntu OS-based servers running in both AWS and Azure clouds. These servers are handling thousands of connections, and we've been experiencing issues with TCP memory usage since upgrading to Ubuntu 22.04.3 from 22.04.2. $ cat /proc/net/sockstat sockets: used 6642 TCP: inuse 5962 orphan 0 tw 292 alloc 6008 mem 128989 UDP: inuse 5 mem 0 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0 As shown in the output below, even after stopping all possible services and closing all open connections, the TCP memory usage remains high and only decreases very slowly. $ cat /proc/net/sockstat sockets: used 138 TCP: inuse 2 orphan 0 tw 0 alloc 3 mem 128320 UDP: inuse 3 mem 0 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0 I have attached a screenshot of linear TCP memory usage growth, which we believe may indicate a TCP memory leak We've never had these issues before, and the only solution we've found so far is to reboot the node. Do you have any suggestions on how to troubleshoot further? Thank you for any help or guidance you can provide! ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-6.2.0-1015-aws 6.2.0-1015.15~22.04.1 ProcVersionSignature: Ubuntu 6.2.0-1015.15~22.04.1-aws 6.2.16 Uname: Linux 6.2.0-1015-aws aarch64 ApportVersion: 2.20.11-0ubuntu82.5 Architecture: arm64 CasperMD5CheckResult: unknown CloudArchitecture: aarch64 CloudID: aws CloudName: aws CloudPlatform: ec2 CloudRegion: us-west-2 CloudSubPlatform: metadata (http://169.254.169.254) Date: Mon Dec 4 13:13:08 2023 Ec2AMI: ami-095a68e28e781dfe1 Ec2AMIManifest: (unknown) Ec2Architecture: arm64 Ec2AvailabilityZone: us-west-2b Ec2Imageid: ami-095a68e28e781dfe1 Ec2InstanceType: m7g.large Ec2Instancetype: m7g.large Ec2Kernel: unavailable Ec2Ramdisk: unavailable Ec2Region: us-west-2 ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash RebootRequiredPkgs: Error: path contained symlinks. SourcePackage: linux-signed-aws-6.2 UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-signed-aws-6.2/+bug/2045560/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
