Bug#926952: sa-exim: Unbuildable/uninstallable in sid
lördag 4 maj 2019 kl. 12:47:18 CEST skrev Magnus Holmgren: > söndag 21 april 2019 kl. 19:55:10 CEST skrev Magnus Holmgren: > > But now that I look closer, it looks like the "spool format error" message > > is only triggered by malformed header files, and Thomas in https:// > > lists.exim.org/lurker/message/20180726.174108.0620f3c0.en.html had > > narrowed > > it down to multiple messages being received over the same connection. I > > still haven't been able to reproduce it though. It might be stochastic. > > Okay, so as I reported on the exim-dev list, I figured out that > primary_hostname, which has been being expanded once and the pointer kept in > static memory since forever, is now being overwritten although allocated > from the permanent pool. My plan is to change this to expand it anew for > each run; that solved the problem when I tested. I'm also going to add an > unrelated patch that disables body rewriting when the spool file is in wire > format and which handles CRLF line endings when scanning headers. Do you > think that you can then re-enable local_scan for the release? Actually, if the only difference wireformat makes is that the line endings are CRLF instead of LF, rewriting the body is probably not a/the problem, as SpamAssassin uses the same line endings as it gets. Carriage returns need to be stripped from the headers, though, so I'll see to that. -- Magnus Holmgrenholmg...@debian.org Debian Developer signature.asc Description: This is a digitally signed message part.
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
söndag 21 april 2019 kl. 19:55:10 CEST skrev Magnus Holmgren: > But now that I look closer, it looks like the "spool format error" message > is only triggered by malformed header files, and Thomas in https:// > lists.exim.org/lurker/message/20180726.174108.0620f3c0.en.html had narrowed > it down to multiple messages being received over the same connection. I > still haven't been able to reproduce it though. It might be stochastic. Okay, so as I reported on the exim-dev list, I figured out that primary_hostname, which has been being expanded once and the pointer kept in static memory since forever, is now being overwritten although allocated from the permanent pool. My plan is to change this to expand it anew for each run; that solved the problem when I tested. I'm also going to add an unrelated patch that disables body rewriting when the spool file is in wire format and which handles CRLF line endings when scanning headers. Do you think that you can then re-enable local_scan for the release? -- Magnus Holmgrenholmg...@debian.org Debian Developer signature.asc Description: This is a digitally signed message part.
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
söndag 21 april 2019 kl. 07:34:15 CEST skrev Andreas Metzler: > On 2019-04-20 Magnus Holmgren wrote: > > tisdag 16 april 2019 kl. 18:46:56 CEST skrev du: > [...] > > > > I have just uploaded exim 4.92-6 to exoerimental which /should/ > > > work with sa-exim, allowing you to debug properly. > > > > Thanks. So far I haven't managed to reproduce the problem of the malformed > > body file, though. > > That is kind of good news, it suggests some temporary bakage in exim > that was fixed since. Or I need to test with more binary data. After all, IIUC, the point of BDAT is that you can send chunks of binary data of given sizes instead of the reveiving server having to look for .. > > However, I produced a segfault in local_scan() when I fed > > exim a spam message using BDAT with report_safe = 1 in local.cf, > > SARewriteBody: 1 in sa-exim.conf and spool_wireformat = true in > > exim4.conf(.template). > > Is this reproducible? I got it multiple times, but then I built and installed a slightly modified sa-exim, and now I can't reproduce it anymore, even after I reinstalled sa- exim from testing. I'm pretty sure that I've been using the same spam mail the entire time too. But I'm thinking that if the problem actually persists, I can at least work around it by checking if body_linecount == 0, which should mean that spool_wireformat = true (or that there's no body to rewrite anyway) and disable body rewriting then. But now that I look closer, it looks like the "spool format error" message is only triggered by malformed header files, and Thomas in https:// lists.exim.org/lurker/message/20180726.174108.0620f3c0.en.html had narrowed it down to multiple messages being received over the same connection. I still haven't been able to reproduce it though. It might be stochastic. -- Magnus Holmgrenholmg...@debian.org Debian Developer signature.asc Description: This is a digitally signed message part.
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
On 2019-04-20 Magnus Holmgren wrote: > tisdag 16 april 2019 kl. 18:46:56 CEST skrev du: [...] > > I have just uploaded exim 4.92-6 to exoerimental which /should/ > > work with sa-exim, allowing you to debug properly. > Thanks. So far I haven't managed to reproduce the problem of the malformed > body file, though. That is kind of good news, it suggests some temporary bakage in exim that was fixed since. > However, I produced a segfault in local_scan() when I fed > exim a spam message using BDAT with report_safe = 1 in local.cf, > SARewriteBody: 1 in sa-exim.conf and spool_wireformat = true in > exim4.conf(.template). Is this reproducible? cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure'
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
tisdag 16 april 2019 kl. 18:46:56 CEST skrev du: > On 2019-04-15 Magnus Holmgren wrote: > > So it seems that everything should work unless spool_wireformat=true > > *and* SARewriteBody: 1. How do I detect in sa-exim whether wire format > > is used for a given body file though? > > -spool_file_wireformat in the -H file? I mean from sa-exim.c. We don't read the -H file directly. > I just do not think the reports of breakage where with > spool_wireformat=true, true though. - Nobody mentioned it and it is not > something one set accidentaly. > > I have just uploaded exim 4.92-6 to exoerimental which /should/ > work with sa-exim, allowing you to debug properly. Thanks. So far I haven't managed to reproduce the problem of the malformed body file, though. However, I produced a segfault in local_scan() when I fed exim a spam message using BDAT with report_safe = 1 in local.cf, SARewriteBody: 1 in sa-exim.conf and spool_wireformat = true in exim4.conf(.template). -- Magnus Holmgrenholmg...@debian.org Debian Developer signature.asc Description: This is a digitally signed message part.
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
On 2019-04-15 Magnus Holmgren wrote: > måndag 15 april 2019 kl. 19:46:14 CEST skrev Andreas Metzler: [l...] > > I have not tested it but I suspect it might break even more when > > spool_wireformat is set. > Isn't it the case that if CHUNKING has been offered AND spool_wireformat is > set, then the body files are in an alternate format? Yes that is true. [...] > So it seems that everything should work unless spool_wireformat=true > *and* SARewriteBody: 1. How do I detect in sa-exim whether wire format > is used for a given body file though? -spool_file_wireformat in the -H file? I just do not think the reports of breakage where with spool_wireformat=true, true though. - Nobody mentioned it and it is not something one set accidentaly. I have just uploaded exim 4.92-6 to exoerimental which /should/ work with sa-exim, allowing you to debug properly. cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure'
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
måndag 15 april 2019 kl. 19:46:14 CEST skrev Andreas Metzler: > Hello, > > Afaiu basically exim has got a new feature and in that case (CHUNKING) > the spoolfile looks a little bit differently and sa-exim fails to handle > this special case properly. So it has BDAT lines in it? > Even in version 4.14 exim's documentation said > > | The file is open for reading and writing, but updating it is not > | recommended > > which I'd read as "you may keep both pieces if it breaks". :-( Well, now that I look closer (it's been too long since I did that) I realise that of couse the headers are available and manipulated through the header_line structs in memory and that it's only the body that's read from and written to the fd, so there should be no problems if SARewriteBody is set to 0 in sa-exim.conf (which is the default). > I have not tested it but I suspect it might break even more when > spool_wireformat is set. Isn't it the case that if CHUNKING has been offered AND spool_wireformat is set, then the body files are in an alternate format? read_message_bdat_smtp() looks like it is like the good old read_message_data_smtp() but for CHUNKING. And the documentation for spool_wireformat says "If this option is set, Exim may for some messages use an alternative format for data-files in the spool which matches the wire format. Doing this permits more efficient message reception and transmission. Currently it is only done for messages received using the ESMTP CHUNKING option." So it seems that everything should work unless spool_wireformat=true *and* SARewriteBody: 1. How do I detect in sa-exim whether wire format is used for a given body file though? -- Magnus Holmgrenholmg...@debian.org Debian Developer signature.asc Description: This is a digitally signed message part.
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
On 2019-04-14 Magnus Holmgren wrote: > fredag 12 april 2019 kl. 19:23:40 CEST skrev du: >> The dlopen localscan patch in exim4 has been nonfunctional in unstable >> for quite some time (4.92~RC2-1/experimental/18 Dec, 4.92~RC3-1 >> unstable/26 Dec and buster/03 Jan). The issue only popped up end of >> March on the upstream user support ML. >> At this time I opened a tracking bug in exim #925982 and looked at >> sa-exim. It is dead upstream since 2006 and buggy: >> * https://lists.exim.org/lurker/message/20180726.113354.6d03efde.en.html >> * #879687 > I've seen reports of bugs, but I didn't realise it was that bad. > sa-exim hasn't exactly changed; has the way local_scan() is called > changed? It's supposed to get an fd, read a message from it, and > possibly write an altered message back (after piping it through > spamc). What can "Format error in spool file" mean? Hello, Afaiu basically exim has got a new feature and in that case (CHUNKING) the spoolfile looks a little bit differently and sa-exim fails to handle this special case properly. Even in version 4.14 exim's documentation said | The file is open for reading and writing, but updating it is not | recommended which I'd read as "you may keep both pieces if it breaks". :-( I have not tested it but I suspect it might break even more when spool_wireformat is set. cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure'
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
fredag 12 april 2019 kl. 19:23:40 CEST skrev du: > The dlopen localscan patch in exim4 has been nonfunctional in unstable > for quite some time (4.92~RC2-1/experimental/18 Dec, 4.92~RC3-1 > unstable/26 Dec and buster/03 Jan). The issue only popped up end of > March on the upstream user support ML. > > At this time I opened a tracking bug in exim #925982 and looked at > sa-exim. It is dead upstream since 2006 and buggy: > * https://lists.exim.org/lurker/message/20180726.113354.6d03efde.en.html > * #879687 I've seen reports of bugs, but I didn't realise it was that bad. sa-exim hasn't exactly changed; has the way local_scan() is called changed? It's supposed to get an fd, read a message from it, and possibly write an altered message back (after piping it through spamc). What can "Format error in spool file" mean? -- Magnus Holmgrenholmg...@debian.org Debian Developer signature.asc Description: This is a digitally signed message part.
Bug#926952: sa-exim: Unbuildable/uninstallable in sid
Package: sa-exim Version: 4.2.1-16 Severity: serious Control: block -1 by 879687 Hello, with exim4/4.92-5 I have dropped support for sa-exim: The dlopen localscan patch in exim4 has been nonfunctional in unstable for quite some time (4.92~RC2-1/experimental/18 Dec, 4.92~RC3-1 unstable/26 Dec and buster/03 Jan). The issue only popped up end of March on the upstream user support ML. At this time I opened a tracking bug in exim #925982 and looked at sa-exim. It is dead upstream since 2006 and buggy: * https://lists.exim.org/lurker/message/20180726.113354.6d03efde.en.html * #879687 (This might be same issue, I just do not know. Chunking is enabled by (upstream) default.) At this point we (exim4) decided that this was a good time to finally drop the dlopen localscan patch. We did that and adopted exim accordingly: - improve the example/docs for content-scanning in exim without sa-exim - drop the abovementioned patch and the virtual Provides for exim4-localscanapi-2.0 and also drop the exim-dev package (only needed for sa-exim). Exim now also Conflicts with sa-exim. However, what I did *not* do was consult you before I did this. I am sorry for that. It is still possible to reinstate exim-dev and try to fix exim to make sa-exim work again. But imho only if sa-exim is fixed in the first place and is tested and found to be working with current exim. What are your thoughts on this? TIA, cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' signature.asc Description: PGP signature
