Bug#644392: dspam truncates mail

2012-04-01 Thread Laurence Withers 
On Sat, Mar 24, 2012 at 12:06:05PM +0100, Julien Valroff wrote:
 Have you been able to test the various commands suggested by Stevan?
 Without answer from you, I will downgrade the severity of this bug.

Dear Julien,

I am sorry, but I have not. Unfortunately the issues I was encountering were
too serious for me to continue experimenting so I had to switch away from
dspam.

Bye for now,
-- 
Laurence Withers, l...@lwithers.me.uk   http://www.lwithers.me.uk/
tel:+447753988197  tel:+44408643   sip:0408...@0408643.call.me.uk



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2012-04-01 Thread Julien Valroff 
Hi Stevan,

Le dimanche 01 avril 2012 à 13:25:19 (+0200 CEST), Stevan Bajić a écrit :
[...]
 @Julien: Can you close this issue?

I've lowered the severity of the bug, but will close it if there is no new
input within the next months or so.

Cheers,
Julien

-- 
  .''`.   Julien Valroff ~ jul...@kirya.net ~ jul...@debian.org
 : :'  :  Debian Developer  Free software contributor
 `. `'`   http://www.kirya.net/
   `- 4096R/ E1D8 5796 8214 4687 E416  948C 859F EF67 258E 26B1



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2012-04-01 Thread Stevan Bajić 

Am 2012-04-01 12:13, schrieb Laurence Withers:

On Sat, Mar 24, 2012 at 12:06:05PM +0100, Julien Valroff wrote:

Have you been able to test the various commands suggested by Stevan?
Without answer from you, I will downgrade the severity of this bug.


Dear Julien,


Hello Laurence,


I am sorry, but I have not. Unfortunately the issues I was 
encountering were
too serious for me to continue experimenting so I had to switch away 
from

dspam.

I still don't think that it is a DSPAM issue but something other. 
Anyway




Bye for now,


@Julien: Can you close this issue?


--
Kind Regards from Switzerland,

Stevan Bajić



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2012-04-01 Thread Thomas Preud'homme 
Le mardi 1 novembre 2011 21:57:04, Stevan Bajić a écrit :

[SNIP]

 
 Looking now more closer to the original message you wrote I realize
 that the message got truncated before the dot. Usually that kind of
 stuff happens if the dot is the first character and it is not properly
 escaped (dot stuffing. See
 http://tools.ietf.org/html/rfc2821#section-4.5.2). But that dot in your
 message is at the end of the line. Somehow I can not believe that it is
 dot stuffing issue. But who knows? It looks like and it smells like a
 dot stuffing issue.

I did a quick check to see where is the dot in the raw message given. The 
answer is it is the byte 1952. This means 1640 bytes were added before the dot 
(1640+1951=3591). This fits with the fact that header could have been added by 
some mail server.

The raw message given in this bug report is 4487 bytes long. With the 1640 
bytes added somewhere before the dot (probably in the header), the total is 
now 6127 bytes, so bigger than the 6021 bytes announced by exim. Besides, the 
mailing list has a footer/signature, there should be at least 198 more bytes 
for the signature.

There is also the -- which is not on a new line in the copy of the mail on 
gmane, I don't know if all this is relevant but I thought it wouldn't hurt to 
put my thought here.

[SNIP]

Thomas


signature.asc
Description: This is a digitally signed message part.

Bug#644392: dspam truncates mail

2012-03-24 Thread Julien Valroff 
Hi Laurence,

Le mardi 01 nov. 2011 à 21:57:04 (+0100 CET), Stevan Bajić a écrit :
[...]
 Can you try to (keep in mind that the message will show up on the
 original date (aka 05.10.2011) in your inbox):
 cat /path/to/the/raw/message/you/sent/me | sendmail -oi l...@lwithers.me.uk
 
 Do you still see the message being truncated?
 
 Can you try to look what DSPAM would do with the message?
 dspam --user l...@lwithers.me.uk --mode=notrain --process
 --deliver=stdout  /path/to/the/raw/message/you/sent/me
 
 Do you see the whole message or do you see a truncation? (I assume
 no truncation because it is a delivery to stdout and not to a
 SMTPD).
 
 Could you post your dspam.conf so that we can see what and how you
 use the delivery agent inside DSPAM?

Have you been able to test the various commands suggested by Stevan?
Without answer from you, I will downgrade the severity of this bug.

Cheers,
Julien

-- 
  .''`.   Julien Valroff ~ jul...@kirya.net ~ jul...@debian.org
 : :'  :  Debian Developer  Free software contributor
 `. `'`   http://www.kirya.net/
   `- 4096R/ E1D8 5796 8214 4687 E416  948C 859F EF67 258E 26B1



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2011-11-01 Thread Laurence Withers 
On Fri, Oct 07, 2011 at 08:46:04AM +0200, Stevan Bajić wrote:
 Do you have the original mail (the 6Kb version) somewhere? In raw
 format? Can you send it to me or post a link to it or attach it to
 this bug report?

Dear Stevan,

I have finally managed to track down a raw copy of the mail, which I have
attached.

I don't believe there is a way to check the size of the mail when exim
delivers it to dspam, unfortunately

Bye for now,
-- 
Laurence Withers, l...@lwithers.me.uk   http://www.lwithers.me.uk/
tel:+447753988197  tel:+44408643   sip:0408...@0408643.call.me.uk
From hema...@ti.com Wed Oct  5 01:58:38 2011
Return-Path: hema...@ti.com
Received: from unknown (HELO bear.ext.ti.com) (192.94.94.41)
  by utopia.booyaka.com with (DHE-RSA-AES256-SHA encrypted) SMTP; 5 Oct 2011 
01:58:38 -
Received: from dbdp20.itg.ti.com ([172.24.170.38])
by bear.ext.ti.com (8.13.7/8.13.7) with ESMTP id p951wTRl031053
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
Tue, 4 Oct 2011 20:58:31 -0500
Received: from dbde70.ent.ti.com (localhost [127.0.0.1])
by dbdp20.itg.ti.com (8.13.8/8.13.8) with ESMTP id p951wRGf012419;
Wed, 5 Oct 2011 07:28:27 +0530 (IST)
Received: from dbde02.ent.ti.com ([172.24.170.145]) by dbde70.ent.ti.com
 ([172.24.170.148]) with mapi; Wed, 5 Oct 2011 07:28:27 +0530
From: Pedanekar, Hemant hema...@ti.com
To: Paul Walmsley p...@pwsan.com
CC: linux-o...@vger.kernel.org linux-o...@vger.kernel.org,
linux-arm-ker...@lists.infradead.org
linux-arm-ker...@lists.infradead.org,
Tony Lindgren t...@atomide.com, Hilman, Kevin khil...@ti.com
Date: Wed, 5 Oct 2011 07:28:25 +0530
Subject: RE: [RFC PATCH v2 0/6] TI81XX: Add clock and hwmod data
Thread-Topic: [RFC PATCH v2 0/6] TI81XX: Add clock and hwmod data
Thread-Index: AcyCd2Lze+DQlKiNQne0MLrf9utx1wAiGsMA
Message-ID: 2a3dcf3da181ad40bde86a3150b27b6b03b4fb3...@dbde02.ent.ti.com
References: 1314119594-22647-1-git-send-email-hema...@ti.com
 alpine.deb.2.00.1110040318260.4...@utopia.booyaka.com
In-Reply-To: alpine.deb.2.00.1110040318260.4...@utopia.booyaka.com
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi Paul,

Paul Walmsley wrote on Tuesday, October 04, 2011 2:54 PM:

 Hi,
 
 On Tue, 23 Aug 2011, Hemant Pedanekar wrote:
 
 This patch set is the v2 of TI816X clock and hwmods patches sent earlier.
 
 The clock data is currently added only for TI816X, while minimal hwmod data
 common for TI816X and TI814X is added.
 
 This patch set depends on following patches:
  TI81XX: Prepare for addition of TI814X support
  TI814X: Add cpu type macros and detection support
  TI814X: Create board support and enable build for TI8148 EVM
 
 Please note that CHIP_IS_TI816X/CHIP_IS_TI814X are still being used and I
 will send update once SoC list usage is available.
 
 I've been spending some time with these patches.  A few aspects don't make
 much sense to me.  For example, looking at the TRMs, the 816x doesn't seem
 to have powerdomains for HDVICP0, 1, or 2, although they are listed in the
 patch.

Can you please check 18.2.1 in TRM from [1].

Above powerdomains are listed there as HDVICP2-0, 1, 2.

  Also, the clockdomain layout is quite different between 814x and
 816x, according to the TRMs.  Are the patches more accurate than the TRMs,
 or vice versa?

You are correct, 814x has differences in clock, clockdomain and powerdomain
data.

I would say the extent of differences level between 816x and 814x is 
something like (please also see reference links from Arago devl tree):

Clock data: ~90% data is different (main contributors: different
PLLs - FAPLL in 816x vs ADPLL 814x, different parent hierarchy)
[2]  [3]

Clockdomains data: ~60% [4]

Powerdomains data: ~90%  [5]
 Will send over the current reworked patches for TI81XX PRCM accessors,
 powerdomain code  data, and clockdomain code  data.  They are intended
 to apply over the TI814x patches that you sent recently.  I'd like to get
 your opinion(s) on these reworked patches.  Please note, so far they have
 only been compile-tested.
 
 
 - Paul

Thanks, I will try those and respond.

   Hemant

[1] 
http://focus.ti.com/general/docs/litabsmultiplefilelist.tsp?literatureNumber=sprugx8
[2] 
http://arago-project.org/git/projects/?p=linux-omap3.git;a=blob;f=arch/arm/mach-omap2/clock816x_data.c;h=32ea958afef915f93e0560f87d1bb2115e24ee2c;hb=HEAD
[3] 
http://arago-project.org/git/projects/?p=linux-omap3.git;a=blob;f=arch/arm/mach-omap2/clock814x_data.c;h=282ffa93398db678843a35766219970655563b46;hb=HEAD
[4] 
http://arago-project.org/git/projects/?p=linux-omap3.git;a=blob;f=arch/arm/mach-omap2/clockdomains81xx.h;h=217c9b4f85c59cc92e0f36ddd73044136c8e9fff;hb=HEAD
[5]

Bug#644392: dspam truncates mail

2011-11-01 Thread Stevan Bajić 

On 01.11.2011 20:57, Laurence Withers wrote:

On Fri, Oct 07, 2011 at 08:46:04AM +0200, Stevan Bajić wrote:

Do you have the original mail (the 6Kb version) somewhere? In raw
format? Can you send it to me or post a link to it or attach it to
this bug report?

Dear Stevan,

Hello Laurence,


I have finally managed to track down a raw copy of the mail, which I have
attached.
thanks. I have already started to look at the file. I don't get that 
6.2Kb size here with the original file but who knows what Exim did to 
the original mail to get about 2Kb more in size than the file you have 
attached? From the past I know that Exim has (had) issues with 8bit 
messages and transformed them into 7bit. Anyway... that original size is 
not the problem.



I don't believe there is a way to check the size of the mail when exim
delivers it to dspam, unfortunately

Okay.


Bye for now,
Looking now more closer to the original message you wrote I realize 
that the message got truncated before the dot. Usually that kind of 
stuff happens if the dot is the first character and it is not properly 
escaped (dot stuffing. See 
http://tools.ietf.org/html/rfc2821#section-4.5.2). But that dot in your 
message is at the end of the line. Somehow I can not believe that it is 
dot stuffing issue. But who knows? It looks like and it smells like a 
dot stuffing issue.


So there are IMHO two possible scenarios (if it is dot stuffing):
1) Exim get's the mail correctly and does not make proper dot stuffing 
when sending it to DSPAM (very unlikely since you use pipe delivery and 
not SMTP)
2) DSPAM got it properly from Exim and has an issue when sending it back 
to Exim


Unfortunately I don't have Exim over here and any test I did, did not 
result in a truncation. Can you on your end try to send that mail again 
into your Exim MTA and look what is happening? Maybe you have the 
possibility to turn on more verbose logging?


Can you try to (keep in mind that the message will show up on the 
original date (aka 05.10.2011) in your inbox):

cat /path/to/the/raw/message/you/sent/me | sendmail -oi l...@lwithers.me.uk

Do you still see the message being truncated?

Can you try to look what DSPAM would do with the message?
dspam --user l...@lwithers.me.uk --mode=notrain --process --deliver=stdout 
 /path/to/the/raw/message/you/sent/me


Do you see the whole message or do you see a truncation? (I assume no 
truncation because it is a delivery to stdout and not to a SMTPD).


Could you post your dspam.conf so that we can see what and how you use 
the delivery agent inside DSPAM?




--
Kind Regards from Switzerland,

Stevan Bajić




--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2011-10-07 Thread Stevan Bajić 

Hello Laurence and Julien,

2011-10-05 01:58:48 1RBGky-0007w4-3k = 
linux-omap-ow...@vger.kernel.org H=mail2.jellyfishnet.co.uk 
[93.91.20.10] P=esmtps X=TLS1.0:RSA_ARCFOUR_MD5:16 S=6021 
id=2a3dcf3da181ad40bde86a3150b27b6b03b4fb3...@dbde02.ent.ti.com


2011-10-05 01:58:48 1RBGky-0007wE-9z = 
linux-omap-ow...@vger.kernel.org U=dspam P=spam-scanned S=3591 
id=2a3dcf3da181ad40bde86a3150b27b6b03b4fb3...@dbde02.ent.ti.com


2011-10-05 01:58:48 1RBGky-0007w4-3k = guralp 
gur...@lwithers.me.uk R=spamscan T=spamcheck



2011-10-05 01:58:48 1RBGky-0007w4-3k Completed



this looks like Exim MTA to me. Right? I am not very fluent in Exim so 
allow me to ask a bunch of questions.


That = and = looks like the direction of the mail flow. = 
indicates inbound while = indicates outbound. Right?


If this is the case then I assume that the mail from 
linux-omap-ow...@vger.kernel.org on 2011-10-05 01:58:48 had a size of 
6021 bytes when received by Exim. Right?


And it was only 3591 bytes when again received by Exim from DSPAM. 
Right?


Is it possible to tell what the size was/is when the mail got delivered 
to DSPAM?




begin routers
# ...
spamscan:
  no_verify
  condition = ${if and {{!eq {$received_protocol}{spam-scanned}} 
{!eq {$received_protocol}{local}} } {1}{0}}

  driver = accept
  transport = spamcheck
  require_files = /var/lib/dspam:+/usr/bin

begin transports
# ...
spamcheck:
  driver = pipe
  command = /usr/bin/dspam --deliver=innocent --user $local_part -- 
%u

  user = dspam
  group = dspam
  return_path_add = false
  log_output = true
  return_fail_output = true


Ahh... yes. I see you mentioning Exim. So it is Exim.

Anyway... when is that spamscan called and when is that spamcheck 
called?


I used piping to DSPAM in the past and had huge issues with it 
(truncating messages and other stuff) and switched to using LMTP 
delivery over a UNIX socket. Never had that kind of issues with LMTP 
delivery. However... that was long time ago (3.4.x/3.6.x).


DSPAM should not change the message content (except adding headers and 
(by request) add a signature (either inline or as an attachment)). But a 
size reduction by factor 2 (almost) is not right. That should not 
happen.


Do you have the original mail (the 6Kb version) somewhere? In raw 
format? Can you send it to me or post a link to it or attach it to this 
bug report?



--
Kind Regards from Switzerland,

Stevan Bajić



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2011-10-05 Thread Laurence Withers 
Package: dspam
Version: 3.10.1+dfs
Severity: critical
Justification: causes serious data loss


I have discovered that dspam is truncating inbound email after it has been
scanned. Please consider the size field (S=) in the following exim log
entries:

2011-10-05 01:58:48 1RBGky-0007w4-3k = linux-omap-ow...@vger.kernel.org 
H=mail2.jellyfishnet.co.uk [93.91.20.10] P=esmtps X=TLS1.0:RSA_ARCFOUR_MD5:16 
S=6021 id=2a3dcf3da181ad40bde86a3150b27b6b03b4fb3...@dbde02.ent.ti.com
2011-10-05 01:58:48 1RBGky-0007wE-9z = linux-omap-ow...@vger.kernel.org 
U=dspam P=spam-scanned S=3591 
id=2a3dcf3da181ad40bde86a3150b27b6b03b4fb3...@dbde02.ent.ti.com
2011-10-05 01:58:48 1RBGky-0007w4-3k = guralp gur...@lwithers.me.uk 
R=spamscan T=spamcheck
2011-10-05 01:58:48 1RBGky-0007w4-3k Completed

As you can see, there is a significant drop in size between the mail being
received from a remote SMTP server (6021 bytes) and after it has been scanned
by DSPAM (3591 bytes).

The mail as it should be is visible here:

http://article.gmane.org/gmane.linux.ports.arm.omap/65281

The mail as I received it ends after the word earlier and immediately
before the period. I can of course provide a copy as written in my .maildir
on request.

-- System Information:
Debian Release: 6.0.2
  APT prefers stable
  APT policy: (1001, 'stable'), (500, 'stable-updates')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2011-10-05 Thread Julien Valroff 
Hi Laurence,

Le mercredi 05 oct. 2011 à 14:49:24 (+0200 CEST), Laurence Withers a écrit :
 Package: dspam
 Version: 3.10.1+dfs
 Severity: critical
 Justification: causes serious data loss
 
 
 I have discovered that dspam is truncating inbound email after it has been
 scanned. Please consider the size field (S=) in the following exim log
 entries:
 
 2011-10-05 01:58:48 1RBGky-0007w4-3k = linux-omap-ow...@vger.kernel.org 
 H=mail2.jellyfishnet.co.uk [93.91.20.10] P=esmtps X=TLS1.0:RSA_ARCFOUR_MD5:16 
 S=6021 id=2a3dcf3da181ad40bde86a3150b27b6b03b4fb3...@dbde02.ent.ti.com
 2011-10-05 01:58:48 1RBGky-0007wE-9z = linux-omap-ow...@vger.kernel.org 
 U=dspam P=spam-scanned S=3591 
 id=2a3dcf3da181ad40bde86a3150b27b6b03b4fb3...@dbde02.ent.ti.com
 2011-10-05 01:58:48 1RBGky-0007w4-3k = guralp gur...@lwithers.me.uk 
 R=spamscan T=spamcheck
 2011-10-05 01:58:48 1RBGky-0007w4-3k Completed
 
 As you can see, there is a significant drop in size between the mail being
 received from a remote SMTP server (6021 bytes) and after it has been scanned
 by DSPAM (3591 bytes).

You use the backport, am I right?

Would you please describe your setup? Do you use DSPAM in daemon mode?

Do you have 'Broken lineStripping' enabled in your dspam.conf?

Cheers,
Julien

-- 
  .''`.   Julien Valroff ~ jul...@kirya.net ~ jul...@debian.org
 : :'  :  Debian Developer  Free software contributor
 `. `'`   http://www.kirya.net/
   `- 4096R/ E1D8 5796 8214 4687 E416  948C 859F EF67 258E 26B1



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#644392: dspam truncates mail

2011-10-05 Thread Laurence Withers 
Hi Julien,

On Wed, Oct 05, 2011 at 06:36:26PM +0200, Julien Valroff wrote:
 You use the backport, am I right?

I believe so; Squeeze backports are enabled. There was a dspam update
maybe a couple of weeks ago or so.

 Would you please describe your setup? Do you use DSPAM in daemon mode?

Sure. I don't use the daemon mode, just the command line agent. Here is an
excerpt from my exim configuration:

begin routers
# ...
spamscan:
  no_verify
  condition = ${if and {{!eq {$received_protocol}{spam-scanned}} {!eq 
{$received_protocol}{local}} } {1}{0}}
  driver = accept
  transport = spamcheck
  require_files = /var/lib/dspam:+/usr/bin

begin transports
# ...
spamcheck:
  driver = pipe
  command = /usr/bin/dspam --deliver=innocent --user $local_part -- %u 
  user = dspam
  group = dspam
  return_path_add = false
  log_output = true
  return_fail_output = true

 Do you have 'Broken lineStripping' enabled in your dspam.conf?

No, it is not enabled. I guess the most important bits of the config (though
you're welcome to the entire thing if you want) are:

Preference spamAction=quarantine
Preference signatureLocation=headers  # 'message' or 'headers'
Preference showFactors=off

Bye for now,
-- 
Laurence Withers, l...@lwithers.me.uk   http://www.lwithers.me.uk/
tel:+447753988197  tel:+44408643   sip:0408643@0408...@call.me.uk



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org