Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Michael Fox
> -Original Message-
> } Unfortunately, it's an amateur radio BBS written in the 80s that
> 
>  Which app is it?

JNOS
http://www.langelaar.net/projects/jnos2/


> } only understands plain text.  It was pretty advanced in its time
> 
>  Something this old may not understand quoted-printable.

It doesn't really understand MIME.  So it spits out quoted-printable as the 
text that it is.

 
>  I have a similar requirement to modify a message where I'm
> using action_external_filter().  My filter() function looks like
> this:

Excellent.  Examples of similar actions (even if they're not directly 
applicable) are a real help to a MIMEDefang and Perl newbie.


> 
> There should probably be some more error checking such as making
> sure that action_external_filter() succeeds.  Perhaps if you look
> at the definition of action_external_filter() in mimedefang.pl it
> will give you some ideas.  It's a fairly simple function (most of
> it is error checking).
> 
> VA7JMN

Thanks.  I'll do that.

Michael
N6MEF


___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread John Nemeth
On Oct 11, 11:54pm, "Michael Fox" wrote:
} 
} > -Original Message-
} > If the old client that you are accommodating is running on (or even just
} > displaying on) a reasonably modern OS, there's a strong chance that
} > you'll actually do better for readability by using UTF-8 as the
} > character set and "8bit" encoding instead of QP. Many (but not all)
} > older mail clients don't really know MIME but do happily pass through
} > all 8 bits of every byte to their output, whether it's a terminal driver
} > or a GUI API.
} 
} Thanks Bill.
} 
} Unfortunately, it's an amateur radio BBS written in the 80s that

 Which app is it?

} only understands plain text.  It was pretty advanced in its time

 Something this old may not understand quoted-printable.

} in that it uses SMTP internally.  But no MIME, no UTF-8, etc.
} It's still extremely useful and widely used, especially in
} emergency comms, delivering mail over VHF radio where other
} connectivity isn't available.

 I have a similar requirement to modify a message where I'm
using action_external_filter().  My filter() function looks like
this:

sub filter () {
my($entity, $fname, $ext, $type) = @_;

return if message_rejected(); # Avoid unnecessary work

if ($fname eq "Cisco_fax.tif") {
#  convert incoming fax to pdf
md_syslog('info', 'converting tiff to pdf');
action_external_filter($entity, "/usr/pkg/bin/tiff2pdf -o 
./FILTEROUTPUT ./FILTERINPUT");
my $head = $entity->head;
$head->replace('content-type', 'application/pdf; name="Cisco_fax.pdf"');
}

return action_accept();
}

There should probably be some more error checking such as making
sure that action_external_filter() succeeds.  Perhaps if you look
at the definition of action_external_filter() in mimedefang.pl it
will give you some ideas.  It's a fairly simple function (most of
it is error checking).

VA7JMN

Note that I've only had licence for a couple of years and am still
getting my feet wet, but I'm definitely interested in data and
emergency comms.  I've recently started playing with RMS Express
(over telnet).  The next thing I want to do is get a TNC.

}-- End of excerpt from "Michael Fox"
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Michael Fox
> -Original Message-
> But you have a tool here that is flexible enough to do it.

Yes, indeed.  

 
> I suppose you want to take the text/plain MIME part and remove any
> other parts. Others would be just an HTML alternative or something
> that is not text.

Yup.  And I'm doing that already, and adding a footer to tell the recipient 
what was removed.  The case I'm trying to solve is when the text/plain or 
text/html part has Content-Transfer-Encoding of Base64.


> If you happen to know that the original was always plain ascii with
> normal line lengths (80 or less) then I'd tempted to say just go from
> base64 to ascii.

Yup.  And "groff -T ascii" has been suggested, just in case there are emoji's 
or other things to squash.  But the quoted-printable suggestion would be less 
destructive so I'm trying that out first.  Either way, 7bit or 
quoted-printable, I apparently need to more carefully parse the MIME structure 
and ultimately rebuild the message.  Hence, my questions.

 
> It does make me wonder why it is base64 at all then, but I've been
> seeing mail from Exchange that is always base64, so maybe that is what
> you are dealing with.

In most cases, it's the carrier or email service provider or sender's client 
app and it's often not something the sending user can control.  So rejecting 
the message results in a complete loss of communications.

Example: send a simple text message like "Hello" from Sprint to an email 
address and they wrap it in HTML and encode it with Base64.

Sending from Verizon, a simple text message arrives as 7bit plain text.  A 
short text, like "Hello", with one emoji arrives as quoted-printable.  A short 
text message with multiple emoji's arrives as base64.  A slightly longer text 
message, like "Me and the kids are safe at Mom's house.  Call when you can.", 
with multiple emoji's arrives as an attachment.  

Different things also happen from smart phone email apps.  So, "there be 
dragons".  And ultimately, this may be beyond my ability to solve.  But I'm not 
ready to give up yet.  If I can convert at least some messages that otherwise 
would not be readable, it's a win.

Michael
a.k.a Don Quixote de La Mancha



___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Michael Fox
> -Original Message-
> I won't give the actual Perl code, but here's a basic sketch, completely
> untested.  I really have no idea whether or not this will work, but I
> think the basic approach is correct. :)

Thanks Dianne.  That definitely points me in the right direction.  Much 
appreciated.

Michael




___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Joseph Brennan
On Fri, Oct 13, 2017 at 9:53 AM, Michael Fox  wrote:
>> -Original Message-
>>
>> >Unfortunately, it's an amateur radio BBS written in the 80s that only
>> understands plain text.
>> > It's still extremely useful and widely used, especially in emergency
>> comms


Interesting case. Most likely none of us have needed to do this type
of conversion. But you have a tool here that is flexible enough to do
it.

I suppose you want to take the text/plain MIME part and remove any
other parts. Others would be just an HTML alternative or something
that is not text.

If you happen to know that the original was always plain ascii with
normal line lengths (80 or less) then I'd tempted to say just go from
base64 to ascii.

It does make me wonder why it is base64 at all then, but I've been
seeing mail from Exchange that is always base64, so maybe that is what
you are dealing with.

Joe Brennan

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Dianne Skoll
On Fri, 13 Oct 2017 10:20:04 -0400
Dianne Skoll  wrote:

>   # non-recursive case: Single part.
>   # If $in_entity is base-64 encoded, make a new $new_entity
>   # that is qp-encoded and call $out_entity->attach($new_entity)
>   # and return 1 to indicate that a change was made.
>   # Otherwise, call $out_entity->attach($in_entity)
>   # and return 0.

Oh, sorry.  ->attach should be ->add_part.

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Dianne Skoll
On Fri, 13 Oct 2017 06:53:33 -0700
"Michael Fox"  wrote:

> While I appreciate everyone's attempt to warn me that "there be
> dragons", the facts are that the client is what it is, it serves
> hundreds of people (in my location alone), and the problem needs
> solving.  

OK.  You will most likely have to do all your changes in
filter_end.  You'll need to traverse the MIME tree, looking for
text/* parts.  If any are Base-64 encoded, you'll need to load them in
and re-encode them using Quoted-Printable.  As you do this, you need
to build up a *new* replacement MIME::Entity.  Finally, call
replace_entire_message with your new entity as the argument.

I won't give the actual Perl code, but here's a basic sketch, completely
untested.  I really have no idea whether or not this will work, but I
think the basic approach is correct. :)

sub re_encode
{
my ($in_entity, $out_entity, $did_something) = @_;

# If it's multipart, recurse
if ($in_entity->is_multipart) {
my $new_entity = MIME::Entity->build(Type => $in_entity->type);
$out_entity->add_part($new_entity);
foreach my $p (@{$in_entity->parts}) {
if (re_encode($p, $new_entity, $did_something)) {
$did_something = 1;
}
}
return $did_something;
}

# non-recursive case: Single part.
# If $in_entity is base-64 encoded, make a new $new_entity
# that is qp-encoded and call $out_entity->attach($new_entity)
# and return 1 to indicate that a change was made.
# Otherwise, call $out_entity->attach($in_entity)
# and return 0.
}

sub filter_end
{
my ($entity) = @_;
my $replacement_entity = MIME::Entity->build(Type => 'multipart/mixed');
my $changes_made = re_encode($entity, $replacement_entity, 0);
if ($changes_made) {
# Recursively remove single-part "multiparts" starting
# from top-level.
while ($replacement_entity->is_multipart && 
scalar(@{$replacement_entity->parts}) == 1) {
$replacement_entity = $replacement_entity->parts(0);
}
replace_entire_message($replacement_entity);
}
}

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Frank Doepper
Am 11.10.17 um 23:54 schrieb Michael Fox:

> Unfortunately, it's an amateur radio BBS written in the 80s that only
> understands plain text.  It was pretty advanced in its time in that it
> uses SMTP internally.  But no MIME, no UTF-8, etc.  It's still extremely
> useful and widely used, especially in emergency comms, delivering mail
> over VHF radio where other connectivity isn't available.

I'd suggest
http://search.cpan.org/~sburke/Text-Unidecode-1.30/lib/Text/Unidecode.pm
and doing it in the receiver's procmailrc if there is any.

Best regards
Frank
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] base64 to quoted-printable

2017-10-13 Thread Michael Fox
> -Original Message-
> 
> >Unfortunately, it's an amateur radio BBS written in the 80s that only
> understands plain text.
> > It's still extremely useful and widely used, especially in emergency
> comms
> 
> So worthwhile investing some cash and getting it updated so that it can
> cope with the modern world?  [clipped]

While I appreciate everyone's attempt to warn me that "there be dragons", the 
facts are that the client is what it is, it serves hundreds of people (in my 
location alone), and the problem needs solving.  

And, while I may be a newbie to MIME and MIMEDefang (and Perl, for that 
matter), I'm definitely NOT a newbie in most other respects.  I've been doing 
network engineering since the days of the NSFnet and I'm pretty certain I 
understand this particular application need better than anyone else here.

So, going forward, I'm hopeful I can get advice and help on MIME and MIMEDefang 
here.  And warnings about dangers of this or that detail or approach are, of 
course, very helpful.  But suggesting that the client be changed doesn't really 
answer the questions that I'm asking.

Paul, Dianne: regarding readability of quoted-printable:  again, I'm a newbie 
to MIMEDefang, not the world.  So I understand and appreciate the warning but I 
still have to solve the problem.  The incoming messages were originated as 
plain ASCII text.  So, the quoted-printable results I'm seeing aren't hard to 
read at all.  But if worse comes to worse, I may end up squashing it to plain 
ASCII text.  We'll see.

Michael



___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang