How many torrents are you seeding, for a point of comparison? I use deluge (headless) on my home server, seeding anywhere from 500- 1000 individual torrents, representing ~5TB of total data, and the process is cruising along at just over 1GB of memory used.
Maybe qbittorrent just doesn't scale well? I feel like I'm on the exceedingly-high end of the spectrum with usage of a torrent program, but I don't know what everyone else is doing.