Re: [Mailman-Users] MM3 on FreeBSD
You may want to contact the current MM2 port maintainer, mandree -at- FreeBSD.org https://www.freshports.org/mail/mailman/ > On Oct 22, 2018, at 09:41, Odhiambo Washington wrote: > > I have finally read bits of documents here and there and managed to install > MM3 on a FreeBSD 11.2 server. This is for testing and getting to grasp the > new monster. Later, I hope I could run my MLs on MM3. > > So, so far, this is what I have done - all successfully: […] -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] admindb web forms not making changes
Hello, I received moved a Mailman instance (via rsync) from a Debian 6 machine to Debian 8, and while most things are generally okay, some the admin web forms don't appear to be working anymore (under Apache 2.4). I've run "check_perms" and it comes back clean. So I can create a new list ("dmtest1") just fine, and the owner gets a message with the new password and such. I then use another e-mail account to send a message to "dmtest1-subscribe" which sends back to corresponding "confirm" e-mail; I click on the confirm URL and that works. The owner / moderator of the list then gets the 'new sub req' message. I click on the "at your conv" link with "admindb/dmtest1", enter the password from the creation message, and I get the list of sub requests. However, when I select "Approve" and click on "Submit All Data", I get sent to the "Dmtest1 Moderator Authentication" page to enter the password. The following appears in Apache's access.log when I click on 'submit': 10.10.10.42 - - [30/Jun/2016:08:52:19 -0400] "POST /mailman/admindb/dmtest1 HTTP/1.1" 200 1268 "http://lists.[foo-example].org/mailman/admindb/dmtest1; "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0" There's nothing in Apache's error.log, and nothing in /usr/local/mailman/logs/. Re-entering the password and approving/submitting is just a loop. Another list admin says that they have some messages with bodies that are bigger than the list limit, and that "discard" does not work. He says that "the page just refreshes". Any ideas? Regards, David -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
On Tue, September 1, 2015 22:43, Mark Sapiro wrote: > On 09/01/2015 06:55 PM, David Magda wrote: >> >> The message in question seems to be from October 2005, and it has the >> "Subject: Você [ ] header. Running under Debian 5 (Python 2.5), archive >> processing wasnt an issue: messages were coming into the mbox and the >> HTML page was being generated until August 25. I then did the upgrade on >> that day. >> >> The mbox then no longer has messages until September 1, the day when I >> did arch wipe, even though the mail.log shows messages coming in (and >> going out AFAICT). > > > So what has probably happened is that archiving has been broken since > the upgrade on Aug 25. There should be a an error message with traceback > in Mailman's 'error' log for each message that wasn't archived. More > importantly, the unarchived messages should all be in Mailman's 'shunt' > queue and running Mailman's bin/unshunt should process them for the > archive. The last error message that is in the 'error' log is from Aug 24: mailman@mail1:~$ ls -lrth logs/ total 16M -rw-rw-r-- 1 mailman mailman 1.2K Mar 2 2014 fromusenet -rw-rw-r-- 1 mailman mailman 63K Mar 18 09:57 locks -rw-rw-r-- 1 www-data mailman 3.1K Mar 31 18:04 mischief -rw-rw-r-- 1 mailman mailman 808K Aug 24 08:00 smtp-failure -rw-rw-r-- 1 mailman mailman 354K Aug 24 08:14 bounce -rw-rw-r-- 1 www-data mailman 1.1M Aug 24 13:14 error -rw-rw-r-- 1 mailman mailman 584K Aug 31 10:45 qrunner -rw-rw-r-- 1 www-data mailman 1.6M Sep 1 12:09 vette -rw-rw-r-- 1 www-data mailman 278K Sep 2 09:03 subscribe -rw-rw-r-- 1 mailman mailman 3.8M Sep 2 09:48 smtp -rw-rw-r-- 1 mailman mailman 2.7M Sep 2 09:48 post >> So would there be anything besides a wipe that could otherwise be >> done to prevent this archive breakage? Attempt a cleanarch" first? >> Seems strange that a message from 2005 could stop archive processing for >> current years/months. > > Presumably, you did some kind of bin/arch process on Aug 25. The answer > is don't do that. Or if you didn't do that, then the answer may lie in > whatever changed in Python. Not that I'm aware of, unless the Debian package runs it in some way pre-/post-upgrade. I didn't start looking at bin/arch until I noticed the breakage. > You should never be running bin/arch without --wipe unless you are > importing archives from elsewhere to be added to an existing archive, > and even in that case, it might be preferable to stop Mailman, > concatenate the existing Mailman mbox and import mbox into a combined > mbox and run 'bin/arch --wipe'. Then after success, start Mailman again. > > You should also consider running the contrib/mmdsr script from the > Mailman source distro or if not that, somehow looking a Mailman's error > log frequently for problems to discover them sooner. We have a few more hosts to upgrade, so I'll have to look more closely at this. However some of these hosts have quite a few lists, so I'm not sure how practical it will be examine every one to see if they're working properly. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
On Tue, September 1, 2015 13:53, Mark Sapiro wrote: > There shouldn't be any non-ascii in a mbox. Well, maybe in a > "Content-Transfer-Encoding: 8bit" (or binary?) body part, but certainly > not in any headers. > > I don't know what you are grepping, but if it's the mbox, you shouldn't > be looking for "\xea", you should be looking for "ê". Yes, there are a number of messages with a C-T-E of 8bit. > Are you running bin/arch with the --wipe option? If not, you are > repeatedly adding the same messages to the archive which is why the > number increases. Doing a run with "--wipe" seems to have fixed things. Doing a complete 're-build' (from message 0) finished without errors, and sending a test message seems to have now generated a proper archive HTML page for September 2015. Previously the last archive web page was for a message sent August 25, even though a few messages had come between then and now. Could the new version of Python been chocking on a binary file whose format has changed from the old version? Is it prudent to do an "arch --wipe" when changing versions of Python? -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
On Tue, September 1, 2015 14:35, Stephen J. Turnbull wrote: > Mark Sapiro writes: > > > I don't know what you are grepping, but if it's the mbox, you shouldn't > > be looking for "\xea", you should be looking for "ê". > > At least on recent BSD-based systems "\xea" is a well-defined escape > sequence, interpreted as the hexadecimal representation of a byte. > Dunno about GNU or proprietary systems. (POSIX.2) This is GNU grep under Debian. > s/just fine/incorrectly but conveniently for the sysadmin/. :-) > > I suppose it's possible that a Python upgrade wiped out a patch or > configuration that told Python to use a Latin-N default encoding, so > it reverted to ASCII. Debian is fairly conservative about these things, and AFAIK, the OS default was to use UTF-8, either en_US or en_CA. The locale did not change during the upgrade. > I suspect that Mailman's copy of the email libraries has also evolved > quite a bit since 2.1.9 (I think that's what you upgraded from?), and > if it was a Mailman provided by the OS vendor, all bets are off. Who > knows what patches they may have applied. We are running 2.1.13 from tarballs and so the Mailman code did not change when the archive web page generation stopped. The only thing that changed was the version of Python (2.5 -> 2.6?) under the OS. Doing a "arch --wipe mylist" seems to have solved the issue, though now I'm curious to know why \xea was a problem before but suddenly isn't after the wipe. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
[Actually send the reply to the list as well.] On Tue, September 1, 2015 12:15, Stephen J. Turnbull wrote: > David Magda writes: > > > When I run 'bin/arch mylistname' I get the following: > > > > [...] > > figuring article archives > > 2005-October > > /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:176: > > UnicodeWarning: Unicode equal comparison failed to convert both arguments > > to Unicode - interpreting them as being unequal > > self.dict = marshal.load(fp) > > /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:74: > > UnicodeWarning: Unicode equal comparison failed to convert both arguments > > to Unicode - interpreting them as being unequal > > self.sorted.sort() > > Updating index files for archive [2004-December] > > [...] > > Updating HTML for article 214 > > Pickling archive state into > > /usr/local/mailman-2.1.20/archives/private/mylistname/pipermail.pck > > Traceback (most recent call last): > > It would appear that you have non-ASCII character in the header of the > 214th message of December 2004 (or maybe it's the 214th message > overall). That message doesn't conform to the mail standards and > should be repaired. > > Since pipermail is constructing an index, I would guess that you have > a localized date header, a display name with an accented character in > it, or a subject with an accented character in it. The character in > question is e with a caret in the Latin-1 set, I don't know if that's > the intended character set though. Looking at the mbox, there was only one place where \xea was in the header, in a Subject line, using `grep --color='auto' -P -n "\xea"`. I manually edited the mbox (making a copy first) and remove the accented-e character with an ASCII "e", and I'm still getting the error (I did this before e-mail the list). There are other places which have \xea, but not in any headers. The 214 is the message count from a state file. Every time I rerun the command the number is higher, but it seems to die in the same place. In the middle of the output we have a "UnicodeWarning": [...] #00104 <...@acm.org> figuring article archives 2005-September #00105 <...@mikep> figuring article archives 2005-September #00106 <...@acm.org> figuring article archives 2005-September #00107 <...@mail.gmail.com> figuring article archives 2005-October /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:176: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal self.dict = marshal.load(fp) /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:74: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal self.sorted.sort() Updating index files for archive [2004-December] Date Subject Author Thread Computing threaded index Updating HTML for article 757 Updating HTML for article 864 Updating HTML for article 866 Updating index files for archive [2005-April] Date [...] Then the error at the end: [...] Updating index files for archive [2005-August] [...] Updating HTML for article 947 Updating HTML for article 840 Updating index files for archive [2005-September] Date Subject Author Thread Computing threaded index Updating HTML for article 841 Updating HTML for article 842 Updating HTML for article 843 Updating HTML for article 952 Updating HTML for article 845 Updating HTML for article 966 Updating HTML for article 846 Updating HTML for article 847 Updating HTML for article 848 Updating HTML for article 957 Updating HTML for article 958 Updating HTML for article 961 Updating HTML for article 962 Updating HTML for article 963 Updating HTML for article 964 Updating HTML for article 965 Updating HTML for article 851 Updating HTML for article 960 Updating HTML for article 861 Updating HTML for article 859 Updating HTML for article 860 Updating HTML for article 970 Pickling archive state into /usr/local/mailman-2.1.20/archives/private/reactome-help/pipermail.pck Traceback (most recent call last): File "bin/arch", line 201, in main() File "bin/arch", line 189, in main archiver.processUnixMailbox(fp, start, end) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 586, in processUnixMailbox self.add_article(a) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 638, in add_article article.parentID = parentID = self.get_parent_info(arch, article) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 658, in get_parent_info if self.database.hasArticle(archive, article.in_reply_to): File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py&qu
Re: [Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
> On Sep 1, 2015, at 21:02, Mark Sapirowrote: > > In my testing with GNU grep on Ubuntu 15.04, 'grep "\xea"' interprets \x > as a literal x and therefore looks for the string "xea", not for the > character whose hex value is EA. For the record/archives: GNU grep also as the “-P” option, which allows Perl regexes (PCRE), and \xhh searches for characters with hex code hh (per pcrepattern(3)): http://stackoverflow.com/questions/3001177/ how-do-i-grep-for-all-non-ascii-characters-in-unix >> Doing a "arch --wipe mylist" seems to have solved the issue, though now >> I'm curious to know why \xea was a problem before but suddenly isn't after >> the wipe. > > > Here's what I suspect was going on. > > Your first run of bin/arch encountered some non-ascii in a header and > threw the exception, but not before writing bad data to the pipermail > database for that month. > > You then "fixed" the non-ascii in the input mbox, but subsequent runs of > bin/arch still encountered the bad data in the database when they got to > that month. > > Finally, you added the --wipe option and that removed everythin and > rebuilt from scratch and as there was no non-ascii in the mbox headers, > it worked. > > As to why this didn't happen before, see my next reply. Sounds plausible. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
> On Sep 1, 2015, at 21:14, Mark Sapiro <m...@msapiro.net> wrote: > > On 09/01/2015 12:09 PM, David Magda wrote: >> >> Could the new version of Python been chocking on a binary file whose >> format has changed from the old version? Is it prudent to do an "arch >> --wipe" when changing versions of Python? > > > The exception is thrown in Python's sort() method. There are no > documented changes in this method's calling sequence between Python 2.5 > and 2.6, but it's quite possible there were changes in the details of > the key comparison code that are responsible. > > No it is not prudent to do an "arch --wipe" when changing versions of > Python?. It is not a good idea to ever do "arch --wipe" unless it's > really necessary. See the section "NOTE ON MBOX ANOMALIES:" at > <http://wiki.list.org/x/4030681>. The message in question seems to be from October 2005, and it has the "Subject: Você […]” header. Running under Debian 5 (Python 2.5), archive processing wasn’t an issue: messages were coming into the mbox and the HTML page was being generated until August 25. I then did the upgrade on that day. The mbox then no longer has messages until September 1, the day when I did “arch —wipe”, even though the mail.log shows messages coming in (and going out AFAICT). So would there be anything besides a “—wipe” that could otherwise be done to prevent this archive breakage? Attempt a “cleanarch" first? Seems strange that a message from 2005 could stop archive processing for current years/months. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
[Not sure if my first attempt made it out.] Hello, We recently upgrade from Debian 5 to 6, and are now having issues with Mailman. Messages are still flowing properly, but the web archives are not being generated since the upgrade. When I run 'bin/arch mylistname' I get the following: [...] figuring article archives 2005-October /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:176: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal self.dict = marshal.load(fp) /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:74: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal self.sorted.sort() Updating index files for archive [2004-December] [...] Updating HTML for article 214 Pickling archive state into /usr/local/mailman-2.1.20/archives/private/mylistname/pipermail.pck Traceback (most recent call last): File "bin/arch", line 201, in main() File "bin/arch", line 189, in main archiver.processUnixMailbox(fp, start, end) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 586, in processUnixMailbox self.add_article(a) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 638, in add_article article.parentID = parentID = self.get_parent_info(arch, article) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 658, in get_parent_info if self.database.hasArticle(archive, article.in_reply_to): File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 279, in hasArticle self.__openIndices(archive) File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 257, in __openIndices t = DumbBTree(os.path.join(arcdir, archive + '-' + i)) File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 66, in __init__ self.load() File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 185, in load self.__sort(dirty=1) File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 74, in __sort self.sorted.sort() UnicodeDecodeError: 'ascii' codec can't decode byte 0xea in position 3: ordinal not in range(128) We're running 2.1.13, but as you can see, the problem persists even with the latest release. We're using the stock Debian Python (2.6.6), but Mailman for tarballs. Thanks for any info. Regards, David -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
Hello, We recently upgrade from Debian 5 to 6, and are now having issues with Mailman. Messages are still flowing properly, but the web archives are not being generated since the upgrade. When I run 'bin/arch mylistname' I get the following: [...] figuring article archives 2005-October /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:176: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal self.dict = marshal.load(fp) /usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:74: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal self.sorted.sort() Updating index files for archive [2004-December] [...] Updating HTML for article 214 Pickling archive state into /usr/local/mailman-2.1.20/archives/private/mylistname/pipermail.pck Traceback (most recent call last): File "bin/arch", line 201, in main() File "bin/arch", line 189, in main archiver.processUnixMailbox(fp, start, end) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 586, in processUnixMailbox self.add_article(a) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 638, in add_article article.parentID = parentID = self.get_parent_info(arch, article) File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line 658, in get_parent_info if self.database.hasArticle(archive, article.in_reply_to): File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 279, in hasArticle self.__openIndices(archive) File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 257, in __openIndices t = DumbBTree(os.path.join(arcdir, archive + '-' + i)) File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 66, in __init__ self.load() File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 185, in load self.__sort(dirty=1) File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line 74, in __sort self.sorted.sort() UnicodeDecodeError: 'ascii' codec can't decode byte 0xea in position 3: ordinal not in range(128) We're running 2.1.13, but as you can see, the problem persists even with the latest release. We're using the stock Debian Python (2.6.6), but Mailman for tarballs. Thanks for any info. Regards, David -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org