Re: longer Debian confession

2003-11-17 Thread Karsten M. Self
on Mon, Nov 17, 2003 at 04:15:50PM +0100, martin f krafft ([EMAIL PROTECTED]) 
wrote:
 Dear all,
 
 A company has asked me whether they [sc]hould write a longer
 confession about their use of Debian, why they chose it, and how
 their impressions are. I asked them to submit a short text to
 www.d.o/users, but they would prefer to write an official statement,
 two pages or so.
 
 They will not publish this text on their own webpage, so the only
 way in which we can profit off this is if there is a place on
 www.d.o to publish it. Is there? Or should I tell them that they
 better spend their time doing other stuff (for Debian) and to stick
 with a 10 line confession in regular www.d.o/users style.

One suggestion is to turn this into an article for publication on a
GNU/Linux news site -- LWN, NewsForge, LinuxToday, IBM's DeveloperWorks,
or others.

If you're looking for a collaborator on such a project, talk to me
offline.


Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
Bush/Cheney '04: The last vote you'll ever have to cast.


pgppXHmwVBSt2.pgp
Description: PGP signature


Re: A case study of a new user turned off debian

2003-11-04 Thread Karsten M. Self
on Mon, Nov 03, 2003 at 04:57:34PM -0500, Greg Stark ([EMAIL PROTECTED]) wrote:
 
 Julian Mehnle [EMAIL PROTECTED] writes:
 
  First, I think what Daniel Jacobowitz said is entirely true. Why didn't you
  start with testing?
 
 Sure testing is less likely to trigger this. 

Frankly, I'd have started stable.  Moving up's tons easier than moving
down.  As you've noted.  And largely trivial.


  Or, you could create a file /etc/apt/preferences and pin the
  testing version of the package with a high enough priority. See
  `man apt_preferences`. Then do a `apt-get dist-upgrade`.
 
 That's about the last place I would send a new user. I read that man
 page about three times during this crisis before I decided it would be
 hopeless to try to explain this procedure online. 

Explain what?

Greg:  Grab this file, copy it to /etc/apt/preferences:
url/preferences
Friend:  OK.

 This is what I meant about there not being an interface. 

File copying, particularly for an experienced sysadmin, is a damned
useful interface.


 If apt said Hm, version 1.2 of libc failed to configure, would you
 like to install the previous version (1.1) from testing and hold back
 the following packages that depend on the new one (awk, grep, sed)
 [Yn]? That would be an interface. 

Pinning can get you much of the way here.

Problem is that dependencies work forward:  you can't be sure of getting
a package in a prior release -- it may not be available.

libc is a particularly pathelogical case, for all the obvious reasons.


 Telling people, go edit this random file with to set pin priorities
 for things to arbitrary numbers, find out what package dependencies
 fail, add those to your list of pin priorities, etc. That's not a
 useful interface for this case.

Telling people to start off with unstable is about as useless.


 In any case having the granularity of stable, testing, unstable
 really doesn't help. All the package versions are in the pool. I want
 to be able to tell apt to try such and such version, or at least put
 back the version I had before and restore whatever other packages it
 must to satisfy dependencies.

Look:  Either KISS and stick with stable, accept the pain of testing, or
learn how to use pinning and come up with a perferences file that works.
Your friend can copy this from any given website you can access.


 I didn't say it was a good idea or that it was going to work. 
 My whole point is that that approach sucks and we should make
 something more effective rather than leave the admin stuck.

Assuming _you_ have experience with Debian, and are aware of limitations
and trade-offs, you've led your friend astray.

I spend a fair amount of time in IRC support for #debian.  Lots of n00bs
come on wanting to install Debian unstable.  The advice is consistent:
Don't.  Not if you don't know your way around tha packaging system yet.
I've had a number of friends convert to Debian -- most of them love it,
aggressively.  But to a one they all wanted to go straight to unstable.
After nearly five years, I still hold to testing with a few exceptions
handled through pins.


Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
Bush/Cheny '04:  BU__SH__!


signature.asc
Description: Digital signature


Re: IMPORTANT: your message to html-tidy

2003-09-09 Thread Karsten M. Self
on Tue, Sep 09, 2003 at 11:07:39AM +1000, Craig Sanders ([EMAIL PROTECTED]) 
wrote:
 On Sun, Sep 07, 2003 at 11:09:57PM -0700, Steve Lamb wrote:
  On Mon, 8 Sep 2003 15:40:15 +1000
  Matthew Palmer [EMAIL PROTECTED] wrote:
   On Mon, Sep 08, 2003 at 06:04:39AM +0100, Karsten M. Self wrote:
I'm coming to the view that we're approaching the era where all mail is
going to have to be subject to filtering, at the MTA level.
   
   Depends on how useful you want your e-mail box to be.  g
  
  It has been my experience that filtering at the MTA level has increased
  the usefulness of my mailbox considerably.  
 
 aol me too /aol
 
 stats from last week's mail.log (from my home mail server which handles mail
 for about half a dozen people):
 
   1   Bad HELO
  10   RBL proxies.relays.monkeys.com
  11   Recipient Domain Not Found
  22   RBL relays.ordb.org
  25   strict 7-bit headers
  31   Relay access denied
  32   RBL taiwan.blackholes.us
  34   Sobig.F Virus
  42   body checks
  49   RBL spamdomains.blackholes.easynet.nl
  56   header checks
  61   RBL dnsbl.sorbs.net
 182   IP Address in HELO
 193   RBL brazil.blackholes.us
 218   RBL blackholes.easynet.nl
 271   Local access rule: Helo command rejected
 342   RBL hongkong.blackholes.us
 492   RBL dynablock.easynet.nl
 924   RBL sbl.spamhaus.org
1080   Local address forgery
1099   Recipient address rejected
1133   Sender Domain Not Found
1771   RBL list.dsbl.org
1825   Dynamic IP Trespass
1902   RBL cn-kr.blackholes.us
2471   Local access rule: Client host rejected
3005   Need FQDN address
3581   Local access rule: Sender address rejected
4267   User unknown
 
   25130   TOTAL
 
 
 Spamassassin stats:
 382   spam
4093   clean
4475   TOTAL
 
 Percentages:
 spam:non-spam (25512/29605) 86.17%
 accepted spam (382/4475) 8.54%
 rejected spam (25130/25512) 98.50%
 
 
 i'm reasonably happy with that.  98.5% of all spam was rejected
 outright.  only 382 spams (1.5%) made it through my postfix access
 lists, RBLs, etc to be tagged by spamassassin.

I'd argue that differently.

You've blocked a total of 6016 mails of 55,117 attempted deliveries,
based on the IP address of the sending MTA's IP address.  That's a broad
rejection policy.  As many people have noted, for pretty much _any_
given IP, your odds are good that most of the mail received from it is
spam.  It doesn't do much for the legit mail that comes through.  Given
that we now _do_ have good content/context based filters for assessing
spam likelihood for a given mail item, blind use of RBLs should be
discouraged.  It's the same sort of thinking that's causing no end of
trouble for people trying to communicate with AOL users:

http://z.iwethey.org/forums/render/content/show?contentid=96264
http://yro.slashdot.org/yro/03/04/13/2215207.shtml?tid=120

I'd recommend alternative approaches -- using RBLs as weighted
indicators, denying first-receipt of mail from such hosts (backing up
their mail queues), 

 these stats also demonstrate just how bad the spam problem has become.
 86% of all attempts to deliver mail to my server were spam, ~25500
 spams and ~4100 legit messages.

No doubt.

 if i wasn't blocking spam at the MTA, then at least half of those
 spams would have ended up in MY personal mailbox (or, more likely,
 tagged by spamassassin and saved into my spam.incoming
 folder)about 13000 more spams than i currently receive.

The difference between what I'm advocating and what you're doing:  run
SpamAssassin _at_ _SMTP_ _receipt_, not after accepting the message for
delivery.  Exim4 allows this readily.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
Charming man, he said. I wish I had a daughter so I could forbid
her to marry one ...
-- HHGTG


pgphCKMNqeC3H.pgp
Description: PGP signature


Re: IMPORTANT: your message to html-tidy

2003-09-08 Thread Karsten M. Self
on Mon, Sep 08, 2003 at 01:57:54PM +1000, Matthew Palmer ([EMAIL PROTECTED]) 
wrote:
 On Sat, Sep 06, 2003 at 04:26:57PM -0700, Joshua Kwan wrote:
  On Sat, Sep 06, 2003 at 06:40:46PM -0400, W3C List Manager wrote:
   This is a response to a message apparently sent from your address to
   [EMAIL PROTECTED]:
   
   Subject: Re: Thank you!
   From:debian-devel@lists.debian.org
   Date:Sat, 6 Sep 2003 18:40:45 --0400
   
   Your message has NOT been distributed to the list; before we distribute 
   it,
   we need your permission to include your message in our Web archive of all
   messages distributed to this list.
  
  How ironic... C-R system at work :)
 
 This one's a bit different.  It's only asking for permission to archive
 posts to the list - I guess W3C's just trying, as hard as possible, to avoid
 any possible legal problems.

It's still an instance in which the autoresponse would not have been
triggered had any half-decent AV/AS system been used to filter out spam
and viruses.  This was a response to the SoBig.F worm.

I'm coming to the view that we're approaching the era where all mail is
going to have to be subject to filtering, at the MTA level.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
In his dream he was walking late at night along the East Side,
beside the river which had become so extravagantly polluted that new
lifeforms were now emerging from it spontaneously, demanding welfare
and voting rights.
-- HHGTG


pgpA3Jo4khB6Y.pgp
Description: PGP signature


Re: IMPORTANT: your message to html-tidy

2003-09-08 Thread Karsten M. Self
on Mon, Sep 08, 2003 at 03:40:15PM +1000, Matthew Palmer ([EMAIL PROTECTED]) 
wrote:
 On Mon, Sep 08, 2003 at 06:04:39AM +0100, Karsten M. Self wrote:
  on Mon, Sep 08, 2003 at 01:57:54PM +1000, Matthew Palmer ([EMAIL 
  PROTECTED]) wrote:
 
 [W3C's autoresponder]
 
   This one's a bit different.  It's only asking for permission to archive
   posts to the list - I guess W3C's just trying, as hard as possible, to 
   avoid
   any possible legal problems.
  
  It's still an instance in which the autoresponse would not have been
  triggered had any half-decent AV/AS system been used to filter out
  spam and viruses.  This was a response to the SoBig.F worm.
 
 Sorry, I didn't make my position sufficiently clear.  This system is
 as broken as every other Challenge-Response, in that it has the
 potential to annoy the shit out of a lot of people very easily, and
 become a nice anonymous harassing agent.
 
 I was just making the point that it isn't the same as a regular C-R
 system, in that the intent wasn't so much to say I want to make sure
 you're not a spammer and more I want to make sure you agree to your
 posts being publically archived - at the very least it's a little
 less offensive than normal (it's not saying You're a spammer - prove
 me wrong!).

Agreed.

This is the difference between broken-by-configuration, and
broken-by-design.  I wasn't saying that the problem was identical to
that of C-R, only that _any_ autoresponder should make reasonable
efforts not to do Joe-Jobs.

MTA behavior can be fixed (or at least greatly remedied) by filtering.
C-R cannot as it assumes the solution to the problem is to offload the
authentication on a third party, itself unverified, unknown,
unauthenticated, and untrusted.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
The truth behind the H-1B IT indentured servant scam:
http://heather.cs.ucdavis.edu/itaa.real.html


pgpWSd5psg75u.pgp
Description: PGP signature


Re: MEI Whitelist Autoresponse

2003-09-01 Thread Karsten M. Self
on Mon, Sep 01, 2003 at 10:51:06AM +1000, Andrew Pollock ([EMAIL PROTECTED]) 
wrote:
 On Sat, Aug 30, 2003 at 01:45:18PM +0300, Richard Braakman wrote:
  
  I think virus scanners are in a different class, though.  Mailing list
  software isn't designed to recognize viruses, while virus scanners are.
  It's disgustingly incompetent to recognize a mail as Sobig.F, which is
  known to fake the sender, and then reply to it anyway.  (And yes, I
  get a lot of notifications that mention Sobig.F by name.)
 
 The (granted, commercial) SMTP virus scanners that I've had experience
 with don't allow you to modify the notification behavior on a per virus
 signature basis, it's either all on or all off.

That is a problem for the vendor.

If the vendor can't modify their software to differentiate, then
notifation should simply be off.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
Scandinavian Designs:  Cool furniture, affordable prices, great service,
satisfied customer.  http://www.scandinaviandesigns.com/


pgpxE1V22AqLV.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-31 Thread Karsten M. Self
on Sat, Aug 30, 2003 at 09:41:39PM -0500, John Hasler ([EMAIL PROTECTED]) wrote:
 I wrote:
  This is about a quarter of my incoming mail.
 
 Karsten writes:
  Which?  Bounces to spoofed senders, or improperly addressed mail?
 
 Bounces.

Thanks.

  What prevents you from 550ing this at SMTP connect?
 
 The absence of any such connections.  I'm on a dialup.

Fair enough.  I'm in the same boat.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   A guide to GNU/Linux browsers:
 http://twiki.iwethey.org/twiki/Main/NixBrowsers


pgpYbdiLuITO6.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-30 Thread Karsten M. Self
on Sat, Aug 30, 2003 at 10:42:17AM +1000, Brian May ([EMAIL PROTECTED]) wrote:
 On Fri, Aug 29, 2003 at 03:48:13PM +1000, Craig Sanders wrote:
  the point that you keep on missing is that TMDA and similar programs send
  confirmation emails to innocent third-parties who did *NOT* send an email.
  
  TMDA and all C-R systems are broken-by-design, just as many stupid end-user
  autoresponders and AV-scanners that send notifications back to the forged
  sender address are broken-by-design.
 
 You saying that any SMTP MTA that sends bounces to unauthenticated
 E-Mail addresses is also broken?

At the very least, this is a small subset of the incoming mail.  There
are probably bad practices, which should be fixed.

The aim is also one which is presumably useful:  if the sender is valid,
then advising them that a message was not delivered is arguably useful
(note that I regard most delivery failure messages as junk).

Most importantly:  the MTA isn't sending mail out willy-nilly to offload
a cost (filtering, content assessment) to a third party.  It's taking an
action on a (hopefully) limited number of mails which cannot be
delivered.

SMTP Envelope reply address should be given precedence, and an SMTP
error precedence over any bounce.

 That is the idea behind autorespoonders after all, to tell the sender
 that his mail didn't get through because it didn't meet some required
 criteria.

The message can't be delivered because of addressing errors is a
different class of error than I can't be bothered to see if this mail
is worth reading, despite its being properly addressed to me.

 Even encryption does not help here, or at least I have not seen any
 proposals for any system that could scale to the Internet. GPG for
 instance only verifies the sender to the receiver, it could not be used
 to verify every sender to the MTAs involved.

A publicly available key, with an email address (or addresses),
validated against contents, is useable.  It doesn't validate the sender,
but it provides a level of indication that someone went through the
trouble of getting a key, posting it publicly, and signing (and/or
encrypting) content with it.

That's more elbow grease than your garden-variety spammer.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   Hollings:  bought, paid for, but couldn't deliver the CBDTPA:
 http://www.politechbot.com/docs/cbdtpa/hollings.s2048.032102.html


pgpea857fP6eC.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-30 Thread Karsten M. Self
on Sat, Aug 30, 2003 at 07:44:56AM -0500, John Hasler ([EMAIL PROTECTED]) wrote:
 Brian May writes:
  You saying that any SMTP MTA that sends bounces to unauthenticated
  E-Mail addresses is also broken?
 
 Karsten M. Self writes:
  At the very least, this is a small subset of the incoming mail.
 
 This is about a quarter of my incoming mail.

Which?  Bounces to spoofed senders, or improperly addressed mail?

What prevents you from 550ing this at SMTP connect?

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   KQED FM:  The bright spot on the dial:  http://www.kqed.org/fm/


pgpIh22S5cRqH.pgp
Description: PGP signature


Re: MEI Whitelist Autoresponse

2003-08-29 Thread Karsten M. Self
on Fri, Aug 29, 2003 at 09:20:53AM +1000, Russell Coker ([EMAIL PROTECTED]) 
wrote:
 On Fri, 29 Aug 2003 07:55, Adam McKenna wrote:
   My own inbox supports this statement.  140 responses to Sobig.F mails,
   of which 43 are virus or other content-based autoresponders, and 97
   being delivery failure messages or other autoresponders (e.g.:  ISP help
   desk).
 
  How many were challenges from mailing list software?  Yes, another class of
  software that automatically issues challenges (specifically, to new
  subscriptions and to non-list members if the list is closed).  So I guess
  you should also file bugs against majordomo, mailman, ezmlm-src, and any
  other mailing list managers that do this.
 
 The comparison to mailing list software makes no sense.
 
 I am prepared to put up with majordomo or mailman responses to virus
 messages because it's for the greater good.  Having a single unwanted
 message go to me is much better than having that message being sent
 out to each of the 10,000 people on a big mailing list!
 
 For challenge-response systems it's totally different.  I don't want
 to receive a single message because a lazy asshole wants to push all
 his problems on other people.
 
 People who take the attitude of Sobig wasn't a problem, my machine
 just sent out 4000 challenge messages to random victims can only be
 described as lazy assholes.

Karsten M. Self repeats the preceding allegations of the Complaint as if
set forth here in full[1].

Mailing lists are a rather small subset of all mail recipients, though
they may be overrepresented in address lists harvested by spammers.

In addition, however, list software _should_ filter virus and spam mail
prior to sending a your message to foo list awaits moderation.

Peace.


Notes:

1.  Someone has been sending too much time reading legal docs.


-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
SCO vs IBM Linux lawsuit info:  http://sco.iwethey.org


pgp79zv54k5Uj.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-29 Thread Karsten M. Self
on Thu, Aug 28, 2003 at 08:21:22AM -0700, Adam McKenna ([EMAIL PROTECTED]) 
wrote:
 On Thu, Aug 28, 2003 at 12:35:25PM +0100, Karsten M. Self wrote:
  #2, Misplaced burden, is the reason for the 'grave' severity.
 
 People have a right to ask that unkown people that e-mail them confirm
 the e-mail.  I'm sorry you don't agree with this, but your opinion is
 hardly justification for a grave bug.

People also have the responsibility to accept, personally, the
responsibility for using, developing, and distributing systems which
impose this burden on others.

If you wish to undertake the distribution of TMDA yourself, with your
own resources, you are free to do so.

You may not demand the right to transfer these consequences on the
Debian Project and SPI over the objections (if present) of the project
at large.  Determining the will of the Debian project is a purpose of
this discussion.

- TMDA should carry a warning to the user about possible consequences
  of activating the C-R mechanism, including sending spam, risking
  blacklisting or registration in spam-reduction services such as
  SpamCop, and a likelihood that some, and perhaps a majority of
  challenges will not be responded to.  The warning should require the
  user to assume full responsibility for doing so.
 
 Sorry, but no.  I will not do this.  The user presumably knows what he is
 installing.

The User demonstrably does not, in all, and possibly in most,
circumstances.

In my own cites of TMDA mailing list experiences, users have apparently
spammed thousands of arbitrary addresses with C-R challenges, and have
found their own systems listed by spam-prevention systems, with neither
the user, nor the architect of TMDA realizing the significance and
externalized costs of TMDA.

Moreover, inclusion of debconf warning messages is *NOT* a
responsibility of upstream, but of the Debian package maintainer.


- Configuration templates for C-R challenges _must_ incorporate virus
  and spam filtering, _prior_ to issuing a C-R challenge.  Preferably,
  tests against obvious header spoofing, if possible, should be
  performed.  Debian tmda packages _must_ depend on corresponding spam
  and virus filters, if this functionality isn't built into TMDA.
  
- Additional strong validation mechanisms, including RFC 2015 PGP
  signed mail and S/MIME signatures, _must_ be used to validate
  sender, including use of web of trust to identify a reasonable
  probability of trusted user status.
  
- If possible, TMDA should be moved to SMTP-time filtering, so that
  mail rejection occurs at SMTP time.  As SMTP doesn't offer a
  protocol for challenge-response, this introduces interesting
  challenges for TMDA's developers.
  
- TMDA's performance _must_ be independently validated and the target
  maximum of 2% challenges to spoofed addresses be confirmed.
  
  
  
  I'm not going to pretend that these are easy fixes.  I'm not a user of
  this package.  I _am_ negatively impacted by it, however, and if it
  continues to display similarly poor consideration of security, abuse,
  and adverse side effects, I fear for Debian, SPI, and the generosity of
  our sponsors.  I do feel the remedies are necessary and advised.  They
  should be communicated upstream, naturally.
 
 I suggest you take these suggestions to the TMDA worker's mailing list at 
 tmda.net, and file wishlist bugs against TMDA for each desired feature.


For what reason can these suggestions not be accomplished (excepting
SMTP-time filtering and independent validation) through providing a
template which applies the proper processing order to a C-R challenge
issuing change, and C-R issuance be disabled unless working AV,
spamfilters, and signature validation SW are installed?  Seems to me
upstream needn't be involved at all.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   Sick of mal-formed websites?  A stylesheet to override poor design:
 http://twiki.iwethey.org/Main/UserContentCSS


pgpmcc4tXb539.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-28 Thread Karsten M. Self
-R functionality.

The problem of false challenges (challenges sent to spoofed headers) is
non-trivial.  It's largely this problem that C-R serves to solve:  the
premise of C-R is that if an email address is deliverable and a human
responds, the address is valid.  So solving the is this a valid address
to test problem is to an extent reflexive.  As an example, my own
address is kmself@ix.netcom.com.  My MTA is guildenstern.dyndns.org.
The netblock this emerges from is an NTL cable line located in the UK.
I myself am on the west coast of the USA.  This is my current correct
mail setup.

The problem can be mitigated.  The question is whether or not this
mitigation is sufficient:

   - TMDA can be seeded with whitelisted domains and addresses.  As
 pointed out, this isn't without its own problems.  See:
 http://lists.debian.org/debian-devel/2003/debian-devel-200308/msg03750.html

   - Virus mails can be filtered out.  This eliminates a large
 percentage of the spoofed headers.  clamav is the only DFSG free
 scanner I find for Debian, though there are proprietary
 alternatives.

   - Spam messages above a reasonable threshold can be filtered out.

This then leaves a small number of messages daily to be assessed -- they
are not viruses, spam, or on an existing whitelist.

My question at this point is:  why not simply look at the damned mail
and figure out for yourself whether or not it's worth reading?  We're
probably talking something like a couple of items, a few times a week.

However, if we want to consider C-R, let's set some guidelines.

A goal would be no challenges to spoofed mails.  Given that this is
nontrivial, we could accept a small number of false positives.

SpamAssassin achieves a false-positive rate (non-spam reported as spam)
of 5% with a default threshold of 5.  This can be dramatically improved
using a whitelist, to ~98% in my experience.  This is not the best
performance of all filters, so makes a somewhat generous threshold.

http://www.spamassassin.org/dist/rules/STATISTICS.txt
http://freshmeat.net/articles/view/964/

So a spam-reduction system user would at worst see a typical rate of 2%
of spam to be manually disposed of.  Should this then be an acceptable
cutoff of spoof challenges -- no more than 2% of challenges be spoofed?
In my response to #4 I showed that I could expect an unnecessary C-R
challenge every four days.  The 2% threshold would reduce this to once
every 200 days, about 6.5 months.  This might be tolerable.



Remedies:

My suggested remedies are as follows:

  - TMDA should be configured, by default, with C-R disabled.  This
appears to be the case currently.

  - TMDA should carry a warning to the user about possible consequences
of activating the C-R mechanism, including sending spam, risking
blacklisting or registration in spam-reduction services such as
SpamCop, and a likelihood that some, and perhaps a majority of
challenges will not be responded to.  The warning should require the
user to assume full responsibility for doing so.

  - Challenges should only be sent as a last resort.

  - Configuration templates for C-R challenges _must_ incorporate virus
and spam filtering, _prior_ to issuing a C-R challenge.  Preferably,
tests against obvious header spoofing, if possible, should be
performed.  Debian tmda packages _must_ depend on corresponding spam
and virus filters, if this functionality isn't built into TMDA.

  - Additional strong validation mechanisms, including RFC 2015 PGP
signed mail and S/MIME signatures, _must_ be used to validate
sender, including use of web of trust to identify a reasonable
probability of trusted user status.

  - If possible, TMDA should be moved to SMTP-time filtering, so that
mail rejection occurs at SMTP time.  As SMTP doesn't offer a
protocol for challenge-response, this introduces interesting
challenges for TMDA's developers.

  - TMDA's performance _must_ be independently validated and the target
maximum of 2% challenges to spoofed addresses be confirmed.



I'm not going to pretend that these are easy fixes.  I'm not a user of
this package.  I _am_ negatively impacted by it, however, and if it
continues to display similarly poor consideration of security, abuse,
and adverse side effects, I fear for Debian, SPI, and the generosity of
our sponsors.  I do feel the remedies are necessary and advised.  They
should be communicated upstream, naturally.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   Data corrupts.  Absolute data corrupts absolutely.
-- Ed Self's corollary of Atkinson's Law.


pgpCCRdJIBBuc.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-28 Thread Karsten M. Self
on Thu, Aug 28, 2003 at 08:56:36AM +0200, Rico -mc- Gloeckner ([EMAIL 
PROTECTED]) wrote:
 On Wed, Aug 27, 2003 at 05:40:46PM -0700, Don Armstrong wrote:
  If possible, perhaps you could consider whitelisting common debian.org
  address by default? [Things like [EMAIL PROTECTED], [EMAIL PROTECTED],
  [EMAIL PROTECTED], etc.]
 
 And would probably defeat the purpose since spammers would know which
 adresses they have to spoof into the From: Header.
 
 Furthermore, if spammers got that, it might happen that they use
 debian.org adresses as sensible default for From: Adresses which will
 raise the amount of Bounces to debian.org. That sounds like a great way
 for the Project to shoot itself into the feet.

That would be an example of #0.  With a twist.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   GNU/Linux web browsing mini review:  Galeon.  Kicks ass.
 http://galeon.sourceforge.org/


pgp4TDCrFo7tX.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-28 Thread Karsten M. Self
on Thu, Aug 28, 2003 at 03:09:48PM +0200, Andreas Metzler ([EMAIL PROTECTED]) 
wrote:
 Karsten M. Self kmself@ix.netcom.com wrote:
 [...]
  SpamAssassin achieves a false-positive rate (non-spam reported as spam)
  of 5% with a default threshold of 5.  This can be dramatically improved
  using a whitelist, to ~98% in my experience.  This is not the best
  performance of all filters, so makes a somewhat generous threshold.
 
 http://www.spamassassin.org/dist/rules/STATISTICS.txt
 http://freshmeat.net/articles/view/964/
 
  So a spam-reduction system user would at worst see a typical rate of 2%
  of spam to be manually disposed of.
 [...]
 
 You are mixing up percentages. 5% non-spam reported as spam ... can
 be ... improved to ~98% ...

Correct.  And yes, I was thinking false-negative.  Spam not flagged as
spam.

What I meant to say was this:

  - Currently feasible content-based filters + whitelists can achieve a
spam rate of 2% of spam passing to the inbox, by independent tests.

  - A C-R system should then target having no more than 2% of challenges
sent be misdirected (based on spoofed headers, etc.).  At this rate,
it's still transferring burden inappropriately, but at a level that
matches a reasonable-case technological alternative.  This also
achieves a secondary goal in the interests of C-R proponents of
keeping the incidence of false challenges low enough that recipients
would be likely to respond to the challenge.


 When I last checked my personal rate with spamassassin 2.55 with
 default rules and no DNS lists or razor (but including a rather well
 trained bayesian filter) and a default threshold of 5, I came up with
 these numbers[1]:
 * 0% false positives, i.e. ham sorted  into the spam folder
 * 10% of the spam was not recognized as such and I had to filter it
   out by hand.

I use a whitelisting system.  It's based on Lars Wizenius's spamfilter
package, my local add being a shell script to scan messages for sender
to add to white, black, gray, or spam lists.  Mail from previously
unknown senders ends up in a grey box.  The principle is the same as
C-R, except that assessment is done by me, rather than a third party.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   Verio webhosting?  Guaranteed downtime:
 http://www.wired.com/news/politics/0,1283,57011,00.html
 http://www.dowethics.com/r/environment/freedom.html


pgp7SQrlsknKk.pgp
Description: PGP signature


Re: Bug#207300: tmda: Challenge-response is fundamentally broken

2003-08-28 Thread Karsten M. Self
on Fri, Aug 29, 2003 at 12:03:34AM +1000, Russell Coker ([EMAIL PROTECTED]) 
wrote:
 On Thu, 28 Aug 2003 21:35, Karsten M. Self wrote:
  Which is a damned good reason for Debian not to package
  viruses and spam mailers.  Or tools which can be readily subverted as
  such.
 
 My Postal program can be used for DOS attacks on mail servers, and has been 
 used for such on at least one occasion (*).

Can be used and is designed to do when used as directed has already
been dealt with and dismissed as a separate case from the one under
consideration.

My understanding of postal is that it is launched at the direction of a
local user.  While this could be embedded into other mechanisms (cgi,
procmail, etc.), it's not packaged or designed specifically for this.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   Spread the real scoop on Xenu and The Church of Scientology, link
   a href=http://xenu.net/;;Scientology/a on your website.


pgpDK1orK2Wyr.pgp
Description: PGP signature


Re: MEI Whitelist Autoresponse

2003-08-28 Thread Karsten M. Self
on Thu, Aug 28, 2003 at 01:03:37AM -0700, Adam McKenna ([EMAIL PROTECTED]) 
wrote:

 Also, I don't have any hard data to support this, but it's obvious to
 me that the volume of mail generated by virus scanners in response to
 Sobig.f eclipses the volume of TMDA challenges by at least a factor of
 10.  So far, I haven't received *one* TMDA challenge that was due to
 Sobig, but I've received *hundreds* of messages from virus scanners
 all over the net.
 
 So, I guess we should add virus scanners to the list of verboten
 software.

My own inbox supports this statement.  140 responses to Sobig.F mails,
of which 43 are virus or other content-based autoresponders, and 97
being delivery failure messages or other autoresponders (e.g.:  ISP help
desk).

The bounces can be reduced.  The virus responses are irresponsible, and
have been for almost two years as the number of sender-spoofing emails
has grown.  I LART a fair number of the responders, report them to
spam-reporting systems, and frequently bounce the mail to the AV
vendor(s) responsible with a nastygram (procmail recipies).

Strongly encouraging virus autoresponders be disabled is also an
independent campaign I've been active in and plan to take to the IT
media mainstream.

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
I managed to love simultaneously -- and this is not easy -- women
and justice.
-- Albert Camus, _The Fall_


pgp0a13OgRHJC.pgp
Description: PGP signature


Re: Fwd: Please confirm your message

2002-12-03 Thread Karsten M. Self
on Sun, Dec 01, 2002 at 07:19:47PM +0100, Gerrit Pape ([EMAIL PROTECTED]) wrote:
 On Sun, Dec 01, 2002 at 02:35:28PM +0100, Russell Coker wrote:
  The people who run such stupid filters misunderstand the way the
  Internet works.
 
 Maybe you should do a short research on the user of this mail handling
 program before saying such.
 
  If you have to send an extra confirmation message every time you send
  an email to someone you haven't communicated with before then it will
  increase the number of messages required by at least 50%.  That is an
  unreasonable burden to place on other people.
 
 I wrote the software primarily for ezmlm mailing lists, please rethink
 your statement with this precondition.

Here's my problem with such tricks:

They take the personal (and best addressed as a personally-managed)
problem of whitelist generation and offload it to a class of people
who neither particularly care, nor are skilled at, executing it.

As is clear here, the tactic is spectacularly ill-suited to mass
communications, mailing lists in particular.  If I'm posting mail to a
list, WTF should I care what Joe Bumpkiss, or Gerrit Pape, wants to do
with my email?  If s/he signed up for the list, the presumption is that
s/he wants to receive the mail.

Ordinarially[1] I use a set of procmail recipies which filter mail on a
number of criteria.  These include heursitics to detect list mail,
spamassassin, and a set of white and black lists.  With my mailer, it's
trivial to select a message, or a list of messages, and add the sender
to either my white or black list.  Takes a fraction of a second.
Only happens once (and generally only for mail directed to me -- list
mail doesn't need this hoop).[2]

Best of all, my system never reveals itself to the sender at all.  Which
is as it should be.

I roundfile any prove yourself requests I receive, and blacklist the
sender.

Peace.



Notes:

1.  System failures mean I'm on a fallback mail system w/o my procmail
support.  Two days of filtering by hand...  I'm going to dig through
backups to get 'em back in place RSN.

2.  The system is based on the Debian spamfilter package, Lars
Wizenius's procmal recipies.  Spamassassin support is simply added
as another rule.  I've added a small script to add an address to a
b/w list.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   Geek for hire:  http://kmself.home.netcom.com/resume.html




Re: Policy proposal: 'local-foo' for system-specific init.d fil

2002-04-02 Thread Karsten M. Self
on Tue, Apr 02, 2002, Osamu Aoki ([EMAIL PROTECTED]) wrote:
 On Sun, Mar 31, 2002 at 10:39:21PM -0800, Sean 'Shaleh' Perry wrote:
  
  On 01-Apr-2002 Henrique de Moraes Holschuh wrote:
   On Sun, 31 Mar 2002, Karsten M. Self wrote:
   This question (where do I put local init.d scripts) comes up enough that
   a policy ought IMVAO be set for it.
   
   Hmm... well, ALL we need in policy is to define that no Debian packages
   should ever use an initscript ID prefixed by local.
   
   Is there reason for more? policy is not a user's guide, after all.
   
  
  the key is to both ensure this behaviour works in Debian (perhaps via 
  policy)
  and have this documented clearly and conspicuously in as many places as
  possible.  The question pops up every 3 weeks or so on the debian-user list.
 
 Actually, this question is answered by FAQ which is hard to find.
 
 http://www.debian.org/doc/FAQ/ch-customizing.html#s-custombootscripts
 
 Problem with FAQ is it is old.  (Potato release days contents)
 
 Policy to prefixed scripts is nice.  FAQ needs to be updated if so.

The only real issue with the answer in the FAQ is that there is a
(slight) chance of a name collision down the road.  Granted, if your
init.d scripts are colliding, there are also probably binaries with
similar names, but

The local-foo policy would avoid this issue.  That and set /usr/local
ahead of /usr and /bin on your search path ;-).

Peace.

-- 
Karsten M. Self kmself@ix.netcom.com   http://kmself.home.netcom.com/
 What Part of Gestalt don't you understand?
   The Consumer Broadband and Digital Television Promotion Act:
 Feinstein's answer to Enron envy.
   http://www.politechbot.com/docs/cbdtpa/hollings.s2048.032102.html


pgpW7rZzh8BEC.pgp
Description: PGP signature


Re: URGENT AND CONFIDENTIAL

2001-09-23 Thread Karsten M. Self
on Sun, Sep 23, 2001 at 01:49:35PM +0100, Oliver Elphick (olly@lfix.co.uk) 
wrote:
 hassan bellob wrote:
   Hassan Bello.
   #20  LOUIS BOTHA CRESCENT
   SADTON SOUTH AFRICA,
   Fax: 874-762-727949. 
   Tel: 874-762-727947.

   {URGENT AND   CONFIDENTIAL} 
   Dear sir,  
   
   In order to transfer out (USD 126 M) One hundred and
   twenty six million United States Dollars) from African
   Development Bank. I have the courage to ask you to
   look for a reliable and honest person who will be
 
 Translation: a complete mug who will fall for this stupid and repetitios
 scam because of the gilded hook dangled in front of him

More info, distribute as needed:

The above is an example of the Nigerian Scam, also called 419 Letters,
or just plain 419.  Most 419 letters and emails originate from or are
traceable back to Nigeria. However, some originate from other nations,
mostly also West African nations such as Ghana, Togo, Liberia, Sierra
Leone, Ivory Coast ( Cote D'Ivoire ) etc.   

It's the current wrinkle of a decades-old scam that nets tens of
millions of dollars per year, and may be the third largest industry in
Nigeria.  The 419 Coalition believes that it is the elites from which
successive governments of Nigeria have been drawn who are the scammers.
Expect little cooperation from Nigerian authorities.

Information in this notice is compiled from various sources, links
below.  Keep reading for reporting information.


The Scam operates as follows: the target receives an unsolicited fax,
email, or letter concerning Nigeria containing either a money laundering
or other illegal proposal OR you may receive a Legal and Legitimate
business proposal by normal means.  Invariably, the proposition involves
resources which are somehow locked up:  oil or other invoice problems, a
bequest, money cleaning, or misallocated funds.

At some point, the victim is asked to pay up front an Advance Fee of
some sort.  If the victim pays the Fee, there are many Complications
which require still more advance payments until the victim either quits,
runs out of money, or both. 


To report the scam:

If you are a United States Citizen or Resident and have suffered NO
Financial Loss, write No Financial Loss - For Your Database on the
documents you received and Fax them to the US Secret Service Task
Force handling Scam matters at 202-406-6930.

Documents may be emailed to 

mailto:[EMAIL PROTECTED]: No Loss -- 419 Scam

IF you are a United States Citizen or Resident and YOU HAVE SUFFERED
A FINANCIAL LOSS write Financial Loss - Contact Me ASAP on the
documents you have received and Fax them to the Task Force at
202-406-6930 and give Your telephone number(s).  A Secret Service
Agent will call you back as soon as possible to discuss the matter
with you (don't worry, you're Not in any trouble).

(Above from http://home.rica.net/alphae/419coal/index.htm)

Other countries have their own reporting methods, see above for more
info.


Additional information:

Google search:
http://www.google.com/search?q=nigeria+scam

Federal Republic of Nigeria, Embassy:
http://www.embassy.org/embassies/ng.html

Sierra Leone:  Nigerian 419 Scam:
http://www.sierra-leone.org/scams.html

Sample letters are archives as no's 01 - 33:
http://www.sierra-leone.org/scam01.html - 
http://www.sierra-leone.org/scam33.html 

Nigeria Scam Letter Archive
http://www.scamorama.com/

Nigeria - The 419 Coalition Website
http://home.rica.net/alphae/419coal/index.htm

U.S. Postal Authorities crack down on Nigerian Spam
http://www.usps.gov/websites/depart/inspect/pressrel.htm

Nigeria - The 419 Coalition Website
http://home.rica.net/alphae/419coal/index.htm

Urban Legends:  Nigerian Scam
http://www.snopes2.com/inboxer/scams/nigeria.htm

Salon: I crave your distinguished indulgence (and all your cash)
http://www.salon.com/people/feature/2001/08/07/419scams/print.html

Nigerian 419 Scam Game Over!
http://home.pacbell.net/jpaladin/
Book describes how the scam plays out.


-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What part of Gestalt don't you understand?  Home of the brave
  http://gestalt-system.sourceforge.net/Land of the free
   Free Dmitry! Boycott Adobe! Repeal the DMCA!  http://www.freesklyarov.org
Geek for Hire  http://kmself.home.netcom.com/resume.html


pgptv0vuhCqPm.pgp
Description: PGP signature


Re: gpg and trustdb very slow

2001-09-22 Thread Karsten M. Self
on Tue, Sep 18, 2001 at 08:37:33AM -0300, Henrique de Moraes Holschuh ([EMAIL 
PROTECTED]) wrote:
 On Tue, 18 Sep 2001, Christian Kurz wrote:
  On 01-09-18 Joey Hess wrote:
   It'd be nice if someone would look at optimizing it sometime; the
   behavior I see with strace is absurd, and could easily be done with no
   syscalls, at least, by just reading the whole trustdb into memory.
  
  I know that the Werner revamped the whole keyring code about 1 or two
  weeks ago. I just tried the --list-keys with my private and the debian
  keyring specified in the options file and I didn't took more then 5
  minutes for me. (AMD K6-200). So I don't think if anyone would really
  want to optimize the code more, he should look at the current cvs code.
 
 As well as reading the docs, where it is said that one must reinsert
 all keys in the trustdb to take advantage of a new caching
 algorithm...

Could you point us to the relevant docs and/or the command for
re-reading keys?

Peace.

-- 
Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/
 What part of Gestalt don't you understand?  Home of the brave
  http://gestalt-system.sourceforge.net/Land of the free
   Free Dmitry! Boycott Adobe! Repeal the DMCA!  http://www.freesklyarov.org
Geek for Hire  http://kmself.home.netcom.com/resume.html


pgpmTtP8aIl64.pgp
Description: PGP signature