Re: [Catalyst] Sending 3000 emails.
On 2015-11-30 9:29 AM, Andrew wrote: ...given that my attempt to send 3000 emails with nothing but my own VPS server, and Cpan's Email::Stuffer, only sent around 1000, before the plack/fastcgi/catalyst app crashed could I have been running into a similar limitation? I'm now thinking of altering my code to split the attempts to send email into six batches of 500 emails, as a short term fix, until I've had time to research which transactional email partner to use, and how their APIs work, or integrate with Catalyst apps. If you are trying to send the emails directly from the process serving your web API, then this is absolutely the wrong approach, and you should give up immediately on trying to do that. Your API process should just queue the messages to be sent by a separate background process, and then return immediately such as with a message "3000 messages queued to be sent soon, click here to check queue progress" or such. Sending in-process may be reasonable for a single email, eg email a copy of the form the user just submitted, but that's it. -- Darren Duncan ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Sending 3000 emails.
On Mon, Nov 30, 2015 at 12:29 PM, Andrew wrote: > I also read on the sendgrid website that: > > "With SMTP, 100 messages can be sent with each connection" > and > "Customers should utilize SMTPAPI if this is an option. As with SMTP, 100 > messages can be sent with each connection, but there can be 1000 recipients > for each message." > > ...given that my attempt to send 3000 emails with nothing but my own VPS > server, and Cpan's Email::Stuffer, only sent around 1000, before the > plack/fastcgi/catalyst app crashed could I have been running into a > similar limitation? I'm now thinking of altering my code to split the > attempts to send email into six batches of 500 emails, as a short term fix, > until I've had time to research which transactional email partner to use, > and how their APIs work, or integrate with Catalyst apps. > > That really depends on whether you're sending directly form your Perl processes (having the perl process connect tot he remote SMTP server, or queuing the messages locally for delivery by a local SMTP demon. In my case, my code hands them to local SMTP, and the local SMPT delivers them upstream (called smarthosting) to sendgrid for delivery to the far corners to the net. > - Original Message - > *From:* Len Jaffe > *To:* The elegant MVC web framework > *Sent:* Sunday, November 29, 2015 1:31 AM > *Subject:* Re: [Catalyst] Sending 3000 emails. > > I'll add that you should look into a transactional email partner like > sendgrid, mailgun, etc. They'll help you stay out of the black lists by > showing you what not to do, ans how to set up DKIM, etc. for authentication > of your email. > > > I use sendgrid professionally, with one of my cusotmers that sends a lot > of email. > > If your volume is under 10k emails per month, sendgrid is free, and they > have a templating feature which allows you to upload a template > (parameterizable) and let them to the mail merge on their servers, for up > to one thousand addresses per tx. > > So instead of generating 3000 emails and pushing them through your smtp > server, you'd upload the template file, and push three 1000-name lists to > to sendgrid and let them take care of delivery. Something to think about. > > Len. > > > On Thu, Nov 26, 2015 at 12:38 PM, Lasse Makholm wrote: > >> Maybe not the answer you're looking for but I'd go with an email service >> such as sendgrid which takes most if not all of the pain out of sending >> emails. They have a simple REST API you can POST your email to and provides >> callbacks for delivery notifications etc as well... >> >> I'm not advocating sendgrid over any other service (I'm sure there are >> plenty others) - it's just the one I have experience with... They >> apparently have a free tier as well >> >> /L >> >> >> On Thu, Nov 26, 2015 at 5:53 PM, Octavian Rasnita >> wrote: >> >>> A queue in Perl is better when there is a need of sending thousand >>> messages. >>> Without a queue, if the Catalyst-based code just sends the messages >>> directly and a browser is waiting for a page to load after the web app sent >>> them, it may time-out. But otherwise it should work and not crash the web >>> app. >>> 3000 messages should be sent pretty fast if they are sent to the local >>> SMTP server, but not fast enough for a pleasant experience. >>> >>> Regarding the success of those messages... if you don't want them to >>> reach in the spam folder, it helps to sign them by using SPF/DKIM. If >>> you'll find this complicated, a better solution might be to create an >>> account on a site like mailgun.com or mandrill.com and send the >>> messages using their servers. They offer APIs that you can use in your app >>> very easy. mailgun.com allows sending 10,000 messages/month for free >>> and after that limit the prices are pretty cheap. You will have reports >>> with the list of email messages that bounced that you can get >>> programaticly, plus a few other helpful features. >>> >>> --Octavian >>> >>> - Original Message - >>> *From:* Andrew >>> *To:* The elegant MVC web framework >>> *Sent:* Thursday, November 26, 2015 5:32 PM >>> *Subject:* [Catalyst] Sending 3000 emails. >>> >>> Hello, >>> >>> You lot are all experts at Catalyst and Perl, >>> and because I'm using Perl to code a Catalyst program, I wanted to ask >>> you about a problem I've encountered. >>> >>> The Catalyst app I'm coding is to replace a PHP website, which presently >>> allows an admin to manage sending out an email to all members. The site has >>> 3000 odd members. From what I could tell, the PHP code was simply using the >>> PHP mail function to send email (although it did in my experience typically >>> take two days for everyone to receive an email sent with it). >>> I've also been asking other Perl programmers when I've had the chance - >>> and the only concerns raised, were about IP blacklisting or emails bouncing >>> back - not about actual code working. >>> >>> So could sending 3000 emails be as simpl
Re: [Catalyst] Sending 3000 emails.
I also read on the sendgrid website that: "With SMTP, 100 messages can be sent with each connection" and "Customers should utilize SMTPAPI if this is an option. As with SMTP, 100 messages can be sent with each connection, but there can be 1000 recipients for each message." ...given that my attempt to send 3000 emails with nothing but my own VPS server, and Cpan's Email::Stuffer, only sent around 1000, before the plack/fastcgi/catalyst app crashed could I have been running into a similar limitation? I'm now thinking of altering my code to split the attempts to send email into six batches of 500 emails, as a short term fix, until I've had time to research which transactional email partner to use, and how their APIs work, or integrate with Catalyst apps. - Original Message - From: Len Jaffe To: The elegant MVC web framework Sent: Sunday, November 29, 2015 1:31 AM Subject: Re: [Catalyst] Sending 3000 emails. I'll add that you should look into a transactional email partner like sendgrid, mailgun, etc. They'll help you stay out of the black lists by showing you what not to do, ans how to set up DKIM, etc. for authentication of your email. I use sendgrid professionally, with one of my cusotmers that sends a lot of email. If your volume is under 10k emails per month, sendgrid is free, and they have a templating feature which allows you to upload a template (parameterizable) and let them to the mail merge on their servers, for up to one thousand addresses per tx. So instead of generating 3000 emails and pushing them through your smtp server, you'd upload the template file, and push three 1000-name lists to to sendgrid and let them take care of delivery. Something to think about. Len. On Thu, Nov 26, 2015 at 12:38 PM, Lasse Makholm wrote: Maybe not the answer you're looking for but I'd go with an email service such as sendgrid which takes most if not all of the pain out of sending emails. They have a simple REST API you can POST your email to and provides callbacks for delivery notifications etc as well... I'm not advocating sendgrid over any other service (I'm sure there are plenty others) - it's just the one I have experience with... They apparently have a free tier as well /L On Thu, Nov 26, 2015 at 5:53 PM, Octavian Rasnita wrote: A queue in Perl is better when there is a need of sending thousand messages. Without a queue, if the Catalyst-based code just sends the messages directly and a browser is waiting for a page to load after the web app sent them, it may time-out. But otherwise it should work and not crash the web app. 3000 messages should be sent pretty fast if they are sent to the local SMTP server, but not fast enough for a pleasant experience. Regarding the success of those messages... if you don't want them to reach in the spam folder, it helps to sign them by using SPF/DKIM. If you'll find this complicated, a better solution might be to create an account on a site like mailgun.com or mandrill.com and send the messages using their servers. They offer APIs that you can use in your app very easy. mailgun.com allows sending 10,000 messages/month for free and after that limit the prices are pretty cheap. You will have reports with the list of email messages that bounced that you can get programaticly, plus a few other helpful features. --Octavian - Original Message - From: Andrew To: The elegant MVC web framework Sent: Thursday, November 26, 2015 5:32 PM Subject: [Catalyst] Sending 3000 emails. Hello, You lot are all experts at Catalyst and Perl, and because I'm using Perl to code a Catalyst program, I wanted to ask you about a problem I've encountered. The Catalyst app I'm coding is to replace a PHP website, which presently allows an admin to manage sending out an email to all members. The site has 3000 odd members. From what I could tell, the PHP code was simply using the PHP mail function to send email (although it did in my experience typically take two days for everyone to receive an email sent with it). I've also been asking other Perl programmers when I've had the chance - and the only concerns raised, were about IP blacklisting or emails bouncing back - not about actual code working. So could sending 3000 emails be as simple as Matt's old CGI form mail? I had a look on CPAN, and found the simple looking Email::Stuffer, and wrote a script, sending an email to myself, and it worked. I then modified the script to pick five email addresses from a MySQL database, and email them, and again - it worked. Okay - one final change - throwing 3000 email addresses at it. How does that do? Starts to work - then throws up an Internal Server Error in the browser window, and I find out m
