On 2025-08-08 06.20, Duncan wrote:
Robin Laing posted on Thu, 7 Aug 2025 23:17:31 -0600 as excerpted:
32G and swap. all used
up. Pan can have access to about 20Gig plus 8 Gig swap. With the large
binary groups, it all gets used up.
Out of curiosity just how many "headers" are we talking? As I mentioned,
pan should handle north of 2 million now, given say 16-gig plus of RAM,
but I really don't know /how/ /far/ north of that ~28 gig (RAM+swap) app-
usable will get, now, nor do I know what sort of numbers such groups
actually have now, so getting some sort of idea to calibrate myself
against modern 64-bit pan and modern high-traffic binary groups would be
useful.
And also... I'm assuming that swap is at least SATA-3 SSD if not M2 or
even compressed-RAM (zswap or swap on zram), and not legacy "spinning
rust", which I can only shudder at the thought of, 8-gig into swap!
Meanwhile, if people are going into swap enough to be a worthwhile
discussion, we could do a subthread on swap and filesystem cache tuning
(basically the stuff under /proc/sys/vm, often configured via /etc/sysctrl
or the like). My system's old enough to be stuck on SATA-3 SSDs still,
but swap and cache tuning have made quite a difference for me.
(FWIW, 16 gig RAM here, gentoo system, and it's stuff like firefox builds
that really send /me/ into swap -- not pan because as I said I don't do
enough binaries for that to be an issue. Chromium builds were even worse
when I was using it, with the various chromium-based webengines near as
bad, and libre-office is supposed to be pretty bad as well. But with
reasonable tuning I can be double-digit gigs into swap and not even
realize it until I happen to glance at my conky memory graphs, while
before tuning I was seeing noticeable swap stalls even a couple hundred MB
into swap.)
Okay, going to start again.
First, found my swap was RAM so my 32G was losing 8G to ram.
Reconfigured system to use a swap file of 23G on an SSD. Machine was
setup many years ago when I didn't think I was going to need a swap file
for anything. Remember the 640K comment for computers? :)
I normally have multiple accounts open to isolate between fun and work
and business. This is normal for me. I am using KDE plasma.
Opened the Fedora version of pan from CLI as pan --debug. Error
messages about missing *.png.
Group I am using for this test is alt.binaries.sleazemovies. As I said,
I like weird that this is a group with weird movies.
On opening. header count is 12626458.
Free went from
free -h
total used free shared buff/cache
available
Mem: 31Gi 7.3Gi 20Gi 195Mi 3.7Gi
24Gi
Swap: 23Gi 0B 23Gi
To
free -h
total used free shared buff/cache
available
Mem: 31Gi 20Gi 3.3Gi 219Mi 8.1Gi
11Gi
Swap: 23Gi 0B 23Gi
Lots of random 723kiB posts that don't mean anything now in this group.
Using search function to sort out and delete some of these headers.
Very slow response.
As Pan is processing the change in the search, the amount of ram being
used has increased. Pan is using 100% of a processor.
total used free shared buff/cache
available
Mem: 31Gi 23Gi 335Mi 220Mi 8.2Gi
8.2Gi
Swap: 23Gi 0B 23Gi
Error message appeared.
(org.gnome.pan:8977): Gtk-CRITICAL **: 23:05:11.991:
gtk_tree_path_compare: assertion 'a != NULL' failed
An interesting thing happened, even though I have NOT selected to update
headers,
I am getting lots of messages about pan (set_watch_mode) and
get_watch_mode) pan/tasks/socket-impl-openssl.cc:7?? and then lots of
repeating statements.
This is a screen cap from the messages after pan stopped.
(/builddir/build/BUILD/pan-v0.163-build/pan-v0.163/pan/tasks/socket-impl-openssl.cc:785:set_watch_mode)
set_watch_mode READ: _tag_watch is now 1512943
(/builddir/build/BUILD/pan-v0.163-build/pan-v0.163/pan/tasks/socket-impl-openssl.cc:742:gio_func)
gio_func: sock 0x7f61400062f0, channel 0x7f614002a650, cond OUT
(/builddir/build/BUILD/pan-v0.163-build/pan-v0.163/pan/tasks/socket-impl-openssl.cc:756:set_watch_mode)
socket 0x7f61400062f0 calling set_watch_mode 2; _channel is 0x7f614002a650
(/builddir/build/BUILD/pan-v0.163-build/pan-v0.163/pan/tasks/socket-impl-openssl.cc:765:set_watch_mode)
channel 0x7f614002a650 setting mode **IGNORE**
(/builddir/build/BUILD/pan-v0.163-build/pan-v0.163/pan/tasks/socket-impl-openssl.cc:785:set_watch_mode)
set_watch_mode IGNORE: _tag_watch is now 0
(/builddir/build/BUILD/pan-v0.163-build/pan-v0.163/pan/tasks/socket-impl-openssl.cc:756:set_watch_mode)
socket 0x7f61400062f0 calling set_watch_mode 0; _channel is 0x7f614002a650
All I did was change the search criteria. Pan now seems frozen. htop
shows 8 processes with one process at ~50% and mem% at 50%
This has gone on for over 10 minutes and I am killing the process.
ctrl+c in the terminal that I started the process to kill it.
didn't just die but slowly closed.
------------------------------
Started pan --debug again. Took 15 minutes to open. 12259817 headers
listed.
With htop open, I am watching the amount of ram increase every second.
I clicked on one article and pan seemed to freeze but was just a slow
response. Removing a large number of articles that are all about 720KiB
in size.
After removing all the random headers I am down to no unread headers.
Closed pan to ensure that the files were updated on the computer and
restarted in --debug again. Very fast open this time.
Getting headers in the group for 800 days. It has been some time since
I have looked at usenet. It is looking like this group is now useless
due to garbage posts.
Trying to scroll through the posts causes pan to freeze. Even though
pan looks like it is using 8 cores, it doesn't seem to change the
percentage used above 100 for one core.
Since this group looks like it is now just garbage, I have stopped the
test at 2383097 headers downloaded and that doesn't even cover all of
today's headers.
Ram usage at this point.
free -h
total used free shared buff/cache
available
Mem: 31Gi 13Gi 14Gi 348Mi 2.8Gi
17Gi
Swap: 23Gi 4.0Ki 23Gi
I will do some more testing at a later date.
If nothing else, I did find an error on my system in regards to swap.
_______________________________________________
Pan-users mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/pan-users
