[Catalyst] Sending 3000 emails.

[Andrew]

Hello,

You lot are all experts at Catalyst and Perl,
and because I'm using Perl to code a Catalyst program, I wanted to ask you 
about a problem I've encountered.

The Catalyst app I'm coding is to replace a PHP website, which presently allows 
an admin to manage sending out an email to all members. The site has 3000 odd 
members. From what I could tell, the PHP code was simply using the PHP mail 
function to send email (although it did in my experience typically take two 
days for everyone to receive an email sent with it).
I've also been asking other Perl programmers when I've had the chance - and the 
only concerns raised, were about IP blacklisting or emails bouncing back - not 
about actual code working.

So could sending 3000 emails be as simple as Matt's old CGI form mail?

I had a look on CPAN, and found the simple looking Email::Stuffer,
and wrote a script, sending an email to myself, and it worked.
I then modified the script to pick five email addresses from a MySQL database, 
and email them, and again - it worked.
Okay - one final change - throwing 3000 email addresses at it.
How does that do?
Starts to work - then throws up an Internal Server Error in the browser window,
and I find out my Fast CGI Plack Up server thingie isn't up, nor running my 
Catalyst app anymore.
I have to type plackup and my details, etc, again from the command prompt.
A log in to Web Host Manager (this is an apache2 VPS with CPanel) and a check 
of the "View Sent Summary" shows 1020 emails sent - 611 successful, 642 
deferrals and 329 failures...curious, as that adds up to more than 1020, =S. If 
I click through for more details, it shows 1582 records - some with green 
ticks, and others with amber or red exclamation marks, ^_^.
I can worry about IPs and failure rates later

I just want to know why the whole FastCGI Plack loaded Catalyst app came 
down half way through,
and how I can have a Perl script process sending 3000 emails without that 
happening every time, =S.

Do I need to use a queue within Perl? Or does the postfix/sendmail on the 
server, automatically queue everything for me? If so - why did my code crash 
the Catalyst app, and not send all the emails? >_<.

I'm guessing this is a common problem - how does a website email all its signed 
up members - and so there must surely be a common solution among Perl / 
Catalyst users?

Any help, appreciated!

Yours,
Andrew.

___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Re: [Catalyst] Sending 3000 emails.

[Bob Miller]

Hi Andrew,


You lot are all experts at Catalyst and Perl,


I am no expert in Catalyst or Perl, but I have learned a thing or two 
about email in my career.


I have found the best way for people to send butt-loads of emails to 
members is to use mailing list software.  This generally involves also 
running a mail server, so it's not a solution from within catalyst or 
perl, but it is almost certainly possible to have a catalyst app manage 
subscribing/unsubscribing/verifying users from a mailing list, and then 
when it comes time to send a mail to everybody, the app only needs to 
send a mail to one list address, and let the mail server do what it does 
best: send mail...


Just a suggestion...


and because I'm using Perl to code a Catalyst program, I wanted to ask
you about a problem I've encountered.
The Catalyst app I'm coding is to replace a PHP website, which presently
allows an admin to manage sending out an email to all members. The site
has 3000 odd members. From what I could tell, the PHP code was simply
using the PHP mail function to send email (although it did in my
experience typically take two days for everyone to receive an email sent
with it).
I've also been asking other Perl programmers when I've had the chance -
and the only concerns raised, were about IP blacklisting or emails
bouncing back - not about actual code working.
So could sending 3000 emails be as simple as Matt's old CGI form mail?
I had a look on CPAN, and found the simple looking Email::Stuffer,
and wrote a script, sending an email to myself, and it worked.
I then modified the script to pick five email addresses from a MySQL
database, and email them, and again - it worked.
Okay - one final change - throwing 3000 email addresses at it.
How does that do?
Starts to work - then throws up an Internal Server Error in the browser
window,
and I find out my Fast CGI Plack Up server thingie isn't up, nor running
my Catalyst app anymore.
I have to type plackup and my details, etc, again from the command prompt.
A log in to Web Host Manager (this is an apache2 VPS with CPanel) and a
check of the "View Sent Summary" shows 1020 emails sent - 611
successful, 642 deferrals and 329 failures...curious, as that adds up to
more than 1020, =S. If I click through for more details, it shows 1582
records - some with green ticks, and others with amber or red
exclamation marks, ^_^.
I can worry about IPs and failure rates later
I just want to know why the whole FastCGI Plack loaded Catalyst app
came down half way through,
and how I can have a Perl script process sending 3000 emails without
that happening every time, =S.
Do I need to use a queue within Perl? Or does the postfix/sendmail on
the server, automatically queue everything for me? If so - why did my
code crash the Catalyst app, and not send all the emails? >_<.
I'm guessing this is a common problem - how does a website email all its
signed up members - and so there must surely be a common solution
among Perl / Catalyst users?
Any help, appreciated!
Yours,
Andrew.


___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/



___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Re: [Catalyst] Sending 3000 emails.

[Octavian Rasnita]

A queue in Perl is better when there is a need of sending thousand messages.
Without a queue, if the Catalyst-based code just sends the messages directly 
and a browser is waiting for a page to load after the web app sent them, it may 
time-out. But otherwise it should work and not crash the web app.
3000 messages should be sent pretty fast if they are sent to the local SMTP 
server, but not fast enough for a pleasant experience.

Regarding the success of those messages... if you don't want them to reach in 
the spam folder, it helps to sign them by using SPF/DKIM. If you'll find this 
complicated, a better solution might be to create an account on a site like 
mailgun.com or mandrill.com and send the messages using their servers. They 
offer APIs that you can use in your app very easy. mailgun.com allows sending 
10,000 messages/month for free and after that limit the prices are pretty 
cheap. You will have reports with the list of email messages that bounced that 
you can get programaticly, plus a few other helpful features.

--Octavian

  - Original Message - 
  From: Andrew 
  To: The elegant MVC web framework 
  Sent: Thursday, November 26, 2015 5:32 PM
  Subject: [Catalyst] Sending 3000 emails.


  Hello,

  You lot are all experts at Catalyst and Perl,
  and because I'm using Perl to code a Catalyst program, I wanted to ask you 
about a problem I've encountered.

  The Catalyst app I'm coding is to replace a PHP website, which presently 
allows an admin to manage sending out an email to all members. The site has 
3000 odd members. From what I could tell, the PHP code was simply using the PHP 
mail function to send email (although it did in my experience typically take 
two days for everyone to receive an email sent with it).
  I've also been asking other Perl programmers when I've had the chance - and 
the only concerns raised, were about IP blacklisting or emails bouncing back - 
not about actual code working.

  So could sending 3000 emails be as simple as Matt's old CGI form mail?

  I had a look on CPAN, and found the simple looking Email::Stuffer,
  and wrote a script, sending an email to myself, and it worked.
  I then modified the script to pick five email addresses from a MySQL 
database, and email them, and again - it worked.
  Okay - one final change - throwing 3000 email addresses at it.
  How does that do?
  Starts to work - then throws up an Internal Server Error in the browser 
window,
  and I find out my Fast CGI Plack Up server thingie isn't up, nor running my 
Catalyst app anymore.
  I have to type plackup and my details, etc, again from the command prompt.
  A log in to Web Host Manager (this is an apache2 VPS with CPanel) and a check 
of the "View Sent Summary" shows 1020 emails sent - 611 successful, 642 
deferrals and 329 failures...curious, as that adds up to more than 1020, =S. If 
I click through for more details, it shows 1582 records - some with green 
ticks, and others with amber or red exclamation marks, ^_^.
  I can worry about IPs and failure rates later

  I just want to know why the whole FastCGI Plack loaded Catalyst app came 
down half way through,
  and how I can have a Perl script process sending 3000 emails without that 
happening every time, =S.

  Do I need to use a queue within Perl? Or does the postfix/sendmail on the 
server, automatically queue everything for me? If so - why did my code crash 
the Catalyst app, and not send all the emails? >_<.

  I'm guessing this is a common problem - how does a website email all its 
signed up members - and so there must surely be a common solution among Perl / 
Catalyst users?

  Any help, appreciated!

  Yours,
  Andrew.



--


  ___
  List: Catalyst@lists.scsys.co.uk
  Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
  Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
  Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Re: [Catalyst] Sending 3000 emails.

[Lasse Makholm]

Maybe not the answer you're looking for but I'd go with an email service
such as sendgrid which takes most if not all of the pain out of sending
emails. They have a simple REST API you can POST your email to and provides
callbacks for delivery notifications etc as well...

I'm not advocating sendgrid over any other service (I'm sure there are
plenty others) - it's just the one I have experience with... They
apparently have a free tier as well

/L

On Thu, Nov 26, 2015 at 5:53 PM, Octavian Rasnita 
wrote:

> A queue in Perl is better when there is a need of sending thousand
> messages.
> Without a queue, if the Catalyst-based code just sends the messages
> directly and a browser is waiting for a page to load after the web app sent
> them, it may time-out. But otherwise it should work and not crash the web
> app.
> 3000 messages should be sent pretty fast if they are sent to the local
> SMTP server, but not fast enough for a pleasant experience.
>
> Regarding the success of those messages... if you don't want them to reach
> in the spam folder, it helps to sign them by using SPF/DKIM. If you'll find
> this complicated, a better solution might be to create an account on a
> site like mailgun.com or mandrill.com and send the messages using their
> servers. They offer APIs that you can use in your app very easy.
> mailgun.com allows sending 10,000 messages/month for free and after that
> limit the prices are pretty cheap. You will have reports with the list of
> email messages that bounced that you can get programaticly, plus a few
> other helpful features.
>
> --Octavian
>
> - Original Message -
> *From:* Andrew 
> *To:* The elegant MVC web framework 
> *Sent:* Thursday, November 26, 2015 5:32 PM
> *Subject:* [Catalyst] Sending 3000 emails.
>
> Hello,
>
> You lot are all experts at Catalyst and Perl,
> and because I'm using Perl to code a Catalyst program, I wanted to ask you
> about a problem I've encountered.
>
> The Catalyst app I'm coding is to replace a PHP website, which presently
> allows an admin to manage sending out an email to all members. The site has
> 3000 odd members. From what I could tell, the PHP code was simply using the
> PHP mail function to send email (although it did in my experience typically
> take two days for everyone to receive an email sent with it).
> I've also been asking other Perl programmers when I've had the chance -
> and the only concerns raised, were about IP blacklisting or emails bouncing
> back - not about actual code working.
>
> So could sending 3000 emails be as simple as Matt's old CGI form mail?
>
> I had a look on CPAN, and found the simple looking Email::Stuffer,
> and wrote a script, sending an email to myself, and it worked.
> I then modified the script to pick five email addresses from a MySQL
> database, and email them, and again - it worked.
> Okay - one final change - throwing 3000 email addresses at it.
> How does that do?
> Starts to work - then throws up an Internal Server Error in the browser
> window,
> and I find out my Fast CGI Plack Up server thingie isn't up, nor running
> my Catalyst app anymore.
> I have to type plackup and my details, etc, again from the command prompt.
> A log in to Web Host Manager (this is an apache2 VPS with CPanel) and a
> check of the "View Sent Summary" shows 1020 emails sent - 611 successful,
> 642 deferrals and 329 failures...curious, as that adds up to more than
> 1020, =S. If I click through for more details, it shows 1582 records - some
> with green ticks, and others with amber or red exclamation marks, ^_^.
> I can worry about IPs and failure rates later
>
> I just want to know why the whole FastCGI Plack loaded Catalyst app
> came down half way through,
> and how I can have a Perl script process sending 3000 emails without that
> happening every time, =S.
>
> Do I need to use a queue within Perl? Or does the postfix/sendmail on the
> server, automatically queue everything for me? If so - why did my code
> crash the Catalyst app, and not send all the emails? >_<.
>
> I'm guessing this is a common problem - how does a website email all its
> signed up members - and so there must surely be a common solution
> among Perl / Catalyst users?
>
> Any help, appreciated!
>
> Yours,
> Andrew.
>
>
> --
>
> ___
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>
>
> ___
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>
>