Re: [racket-users] Racket application servers

[Brian Adkins]

I finally got a very simple, "hello world", Racket web app up and running, 
and I'm very encouraged with the performance. I just started a single 
Racket instance and proxy to it from nginx. I have it running on an AWS EC2 
instance, and running the Apache Benchmark (ab) utility on my laptop as 
follows reported 300 requests per second (38 concurrent requests, 1000 
requests total):

ab -c 38 -n 1000 http://hello-app

Granted, the "hello world" app is doing the bare minimum (no database 
connection, minimal processing, etc., but it is logging requests), but the 
fact that I'm getting 300 req/sec including proxying through nginx with one 
single-threaded Racket process (& zero time spent tuning) is great news! Of 
course, over a decade with Ruby may have lowered my performance bar 
somewhat ;)

I can't wait to get a more representative test running.

On Monday, November 26, 2018 at 10:38:23 AM UTC-5, Brian Adkins wrote:
>
> The current Ruby/Rails app will max out at least one core at times now. I 
> realize Racket should be faster, but I expect I'll still need more than a 
> single core for the app as the volume will be going up significantly in 
> January, and as you mentioned, there are some other benefits to a 
> multi-process architecture.
>
> On Sunday, November 25, 2018 at 11:26:11 AM UTC-5, Greg Hendershott wrote:
>>
>> I just want to point out the possibility that your Racket web app 
>> might not be CPU-bound. Some "generic" web sites are IO-bound. Blocked 
>> on IO for the HTTP requests and responses. Blocked on IO talking to a 
>> database server like Postgres. 
>>
>> In cases like that, you might not need more than one process. (Indeed 
>> you might even get away with running on a t2.micro instance, which 
>> throttles horribly after a short CPU burst, because you never come 
>> close to that threshold even under your maximum traffic loads.) 
>>
>> It's a real possibility you might want to measure/see, first. Of 
>> course it depends on how much work your Racket web app does (itself, 
>> not farmed out to DB or other out-of-process servers), as well as on 
>> the traffic loads you expect. 
>>
>> Having two or more servers might be convenient for non-load reasons. 
>> For updates (to let the old "drain" as you described, or blue/green 
>> deploys, etc.).  Or for fail-over (although I'm not sure 2 procs on 
>> same box is the way to go, if you even really need many 9s (many sites 
>> really don't if we're being honest with ourselves)). 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Brian Adkins]

Very interesting. I'll check out your project - thanks for mentioning it.

On Monday, November 26, 2018 at 4:42:15 AM UTC-5, Jérôme Martin wrote:
>
> Just so you know, I started some months ago a Racket project that would 
> help monitor different Racket web servers and load-balance them, using 
> systemd-nspawn as a native container system, and an optional
>  nginx server to load balance everything, serve static content and cache 
> dynamic one.
>
> It's called Bonny: https://github.com/euhmeuh/bonny
>
> It's in a very early stage right now but I'm actively working on it so 
> that I can deploy and monitor my servers without having to rely on some big 
> overkill container app du jour.
>
> I'd be really glad to talk about some approaches you use and what seems 
> the easiest to implement. My main objectives are minimalism, use as much 
> native systems as possible, and as much Racket as possible.
>
> Plus, containers in Bonny are called "pirates" (cause you're "shipping 
> them", ahah).
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Brian Adkins]

The current Ruby/Rails app will max out at least one core at times now. I 
realize Racket should be faster, but I expect I'll still need more than a 
single core for the app as the volume will be going up significantly in 
January, and as you mentioned, there are some other benefits to a 
multi-process architecture.

On Sunday, November 25, 2018 at 11:26:11 AM UTC-5, Greg Hendershott wrote:
>
> I just want to point out the possibility that your Racket web app 
> might not be CPU-bound. Some "generic" web sites are IO-bound. Blocked 
> on IO for the HTTP requests and responses. Blocked on IO talking to a 
> database server like Postgres. 
>
> In cases like that, you might not need more than one process. (Indeed 
> you might even get away with running on a t2.micro instance, which 
> throttles horribly after a short CPU burst, because you never come 
> close to that threshold even under your maximum traffic loads.) 
>
> It's a real possibility you might want to measure/see, first. Of 
> course it depends on how much work your Racket web app does (itself, 
> not farmed out to DB or other out-of-process servers), as well as on 
> the traffic loads you expect. 
>
> Having two or more servers might be convenient for non-load reasons. 
> For updates (to let the old "drain" as you described, or blue/green 
> deploys, etc.).  Or for fail-over (although I'm not sure 2 procs on 
> same box is the way to go, if you even really need many 9s (many sites 
> really don't if we're being honest with ourselves)). 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Jérôme Martin]

Just so you know, I started some months ago a Racket project that would 
help monitor different Racket web servers and load-balance them, using 
systemd-nspawn as a native container system, and an optional
 nginx server to load balance everything, serve static content and cache 
dynamic one.

It's called Bonny: https://github.com/euhmeuh/bonny

It's in a very early stage right now but I'm actively working on it so that 
I can deploy and monitor my servers without having to rely on some big 
overkill container app du jour.

I'd be really glad to talk about some approaches you use and what seems the 
easiest to implement. My main objectives are minimalism, use as much native 
systems as possible, and as much Racket as possible.

Plus, containers in Bonny are called "pirates" (cause you're "shipping 
them", ahah).

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Greg Hendershott]

I just want to point out the possibility that your Racket web app
might not be CPU-bound. Some "generic" web sites are IO-bound. Blocked
on IO for the HTTP requests and responses. Blocked on IO talking to a
database server like Postgres.

In cases like that, you might not need more than one process. (Indeed
you might even get away with running on a t2.micro instance, which
throttles horribly after a short CPU burst, because you never come
close to that threshold even under your maximum traffic loads.)

It's a real possibility you might want to measure/see, first. Of
course it depends on how much work your Racket web app does (itself,
not farmed out to DB or other out-of-process servers), as well as on
the traffic loads you expect.

Having two or more servers might be convenient for non-load reasons.
For updates (to let the old "drain" as you described, or blue/green
deploys, etc.).  Or for fail-over (although I'm not sure 2 procs on
same box is the way to go, if you even really need many 9s (many sites
really don't if we're being honest with ourselves)).

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Brian Adkins]

I probably wasn't very clear about what's going on with Unicorn when I 
deploy a new version of the app. It's actually quite simple, and doesn't 
require loading new code on the fly. Here's what I do:

1) I update the app server code via:  git pull  (this doesn't affect the 
running processes due to Linux inode handling)
2) I send a signal to the Unicorn master process via:  kill -USR2 
3) Unicorn then instructs each of the worker processes to quit after 
finishing their current request, if any.
4) When each worker process quits, Unicorn automatically spins up a new one 
that happens to use the new code

Requests continue to be handled by existing worker processes, so there's no 
downtime.

I'm not too concerned about this aspect - I could easily instrument this 
manually by making the worker processes accept a signal/message/etc. that 
would instruct them to quit after the current request.

The monit solution seems fine, but it typically works via a pid file, so 
I'd simply have to ensure that the N Racket server processes write their 
respective pid's to N separate files. It's not a big deal, but having 
something like Unicorn handle that is handy. 

I could probably just have monit pass a unique argument to each of the N 
processes e.g. 1, 2, 3, ... , N and store the pid's in pid1, pid2, ... , 
pidN. They could also use the arg to form the port they listen on 4311, 
4312, ... , 431N

I'm probably going to be dealing with only 2 or 4 cores typically, so the 
final solution doesn't need to be all that complicated.

I appreciate all the info folks have supplied, and I learned something new 
from George about multiple apps listening on the same port - I assume that 
involves SO_REUSE - I may have to research that. Having the OS handle the 
scheduling may be worth considering.

On Saturday, November 24, 2018 at 2:21:11 PM UTC-5, Jesse Alama wrote:
>
> Hi Brian, 
>
> On 23 Nov 2018, at 21:46, Brian Adkins wrote: 
>
> > I'm porting a web application from Ruby/Rails to Racket, and I'd like 
> > something to manage the Racket server processes. 
> > 
> > In the Ruby world, I'm currently using Unicorn ( 
> > https://en.wikipedia.org/wiki/Unicorn_(web_server) ) prior to that I 
> > used 
> > Nginx Passenger ( https://en.wikipedia.org/wiki/Phusion_Passenger ), 
> > etc. 
> > Another popular Ruby app server is Puma 
> > ( https://en.wikipedia.org/wiki/Puma_(web_server) ) 
> > 
> > I'll use nginx as the front end web server, and it will proxy to the 
> > application server. In a nutshell, for Unicorn, I configure the 
> > location of 
> > the Rails app, configure how many processes I want, and Unicorn will 
> > spin 
> > up that number of Rails processes and handle routing requests from 
> > nginx to 
> > each of the Rails processes in some fashion. If a Rails process exists 
> > abnormally, it will spin up another one to replace it. To deploy a new 
> > version of my app, I can send a signal to the Unicorn master process 
> > to 
> > *gracefully* restart all the processes i.e. it waits for the current 
> > request to finish, and then kills the process and spins up a new one 
> > using 
> > the new version of the app. 
> > 
> > Are there similar application servers available for Racket? 
> > Alternatively, 
> > if not, and you have long running applications in Racket, what are you 
> > using to manage them? 
>
> Just to echo the experience of others here: I've also used the Racket 
> server straight up on the unfiltered Internet, and had no problems 
> except (as Greg mentioned) weird requests flowing in that the server 
> found dubious. Without crashing, I should add; these weird (if not 
> downright malformed) requests didn't kill the web server, they just 
> created a lot of junk in the logs. Putting Racket behind a server that's 
> more prepared to handle such junk (e.g., OpenBSD's httpd) restores log 
> sanity. 
>
> To get back to your real question, though: you want to know how to 
> deploy a new version of a running web app. 
>
> (1) There's the reloadable package 
>
> https://pkgs.racket-lang.org/package/reloadable 
>
> I myself haven't used it, but it's definitely relevant. That looks like 
> a real Racket-y solution, which is what I think you're looking for. 
>
> (2) I myself view these issues as DevOps-y things to be dealt with not 
> so much in Racket and more at the OS/service level. Generating an 
> httpd.conf file and then using OpenBSD's rcctl to switch out the old 
> config for the new one has been fine for me. This is perhaps a deviant 
> view; the reloadable package probably reflects a more catholic 
> understanding of the situation. But there may be cases where you change 
> your code in ways that are just too big, and I bet you'll find yourself 
> needing to do something like what I do and what you're currently doing 
> with Unicorn. 
>
> Jesse 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving 

Re: [racket-users] Racket application servers

[Jesse Alama]

Hi Brian,

On 23 Nov 2018, at 21:46, Brian Adkins wrote:


I'm porting a web application from Ruby/Rails to Racket, and I'd like
something to manage the Racket server processes.

In the Ruby world, I'm currently using Unicorn (
https://en.wikipedia.org/wiki/Unicorn_(web_server) ) prior to that I 
used
Nginx Passenger ( https://en.wikipedia.org/wiki/Phusion_Passenger ), 
etc.

Another popular Ruby app server is Puma
( https://en.wikipedia.org/wiki/Puma_(web_server) )

I'll use nginx as the front end web server, and it will proxy to the
application server. In a nutshell, for Unicorn, I configure the 
location of
the Rails app, configure how many processes I want, and Unicorn will 
spin
up that number of Rails processes and handle routing requests from 
nginx to

each of the Rails processes in some fashion. If a Rails process exists
abnormally, it will spin up another one to replace it. To deploy a new
version of my app, I can send a signal to the Unicorn master process 
to

*gracefully* restart all the processes i.e. it waits for the current
request to finish, and then kills the process and spins up a new one 
using

the new version of the app.

Are there similar application servers available for Racket? 
Alternatively,

if not, and you have long running applications in Racket, what are you
using to manage them?


Just to echo the experience of others here: I've also used the Racket 
server straight up on the unfiltered Internet, and had no problems 
except (as Greg mentioned) weird requests flowing in that the server 
found dubious. Without crashing, I should add; these weird (if not 
downright malformed) requests didn't kill the web server, they just 
created a lot of junk in the logs. Putting Racket behind a server that's 
more prepared to handle such junk (e.g., OpenBSD's httpd) restores log 
sanity.


To get back to your real question, though: you want to know how to 
deploy a new version of a running web app.


(1) There's the reloadable package

https://pkgs.racket-lang.org/package/reloadable

I myself haven't used it, but it's definitely relevant. That looks like 
a real Racket-y solution, which is what I think you're looking for.


(2) I myself view these issues as DevOps-y things to be dealt with not 
so much in Racket and more at the OS/service level. Generating an 
httpd.conf file and then using OpenBSD's rcctl to switch out the old 
config for the new one has been fine for me. This is perhaps a deviant 
view; the reloadable package probably reflects a more catholic 
understanding of the situation. But there may be cases where you change 
your code in ways that are just too big, and I bet you'll find yourself 
needing to do something like what I do and what you're currently doing 
with Unicorn.


Jesse

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Brian Adkins]

I'm glad to hear about the Racket web server's reliability! I'm still 
working through the high level architecture, but it seems like one Racket 
instance per core make sense to maximize cpu utilization, and then I may 
spin up a thread per request w/in each process to maximize memory 
utilization.

I like using nginx in front of the application for a couple reasons: 1) It 
handles non-application requests (e.g. simple files) very fast, and 2) I'm 
very familiar with configuring it for multiple distinct apps, SSL, etc.

So, in the absence of a Unicorn or Puma-like app server in Racket, I think 
a couple good options are:

1) Use monit to manage a set of Racket server processes (one per core), and 
have nginx load balance requests to them (or alternatively, as George 
mentioned, let the OS load balance by having the server processes all 
listen on the same port).

2) Write a simple master process (i.e. very scaled down Unicorn or Puma) in 
Racket that manages a Racket worker process per core.

I'm pretty sure that I'll eventually want option #2, but I may need to go 
with #1 initially in the interest of time.

On Saturday, November 24, 2018 at 12:03:27 AM UTC-5, Greg Hendershott wrote:
>
> In my experience a Racket web server will just stubbornly continue to 
> work for months at a time, if you let it. 
>
> (Reminds me of the aviation joke. In the future, cockpits will have 
> just one human pilot and a dog. The dog is there to bite the human if 
> they try to turn off the autopilot.) 
>
> I have a site on one little t2.micro at AWS. 
>
> It is behind an AWS application load balancer -- but mainly just as an 
> easy way to do SSL termination. 
>
> I also added WAF to filter script kiddies who don't bother to supply a 
> valid Host header while they try to log into phpadmin or whatever. For 
> non-FANG sites honestly that will be a large proportion of your 
> traffic in terms of number of requests. It wasn't making the Racket 
> web server sweat, I just wanted to noise out of my logs and stats. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Philip McGrath]

I'm not sure if this was clear from what I said earlier, but I use the
Racket web server without anything in front of it. In fact, Racket does SSL
termination and serves as a proxy for non-Racket services. We have been
completely satisfied with its performance and reliability.

-Philip


On Sat, Nov 24, 2018 at 12:03 AM Greg Hendershott 
wrote:

> In my experience a Racket web server will just stubbornly continue to
> work for months at a time, if you let it.
>
> (Reminds me of the aviation joke. In the future, cockpits will have
> just one human pilot and a dog. The dog is there to bite the human if
> they try to turn off the autopilot.)
>
> I have a site on one little t2.micro at AWS.
>
> It is behind an AWS application load balancer -- but mainly just as an
> easy way to do SSL termination.
>
> I also added WAF to filter script kiddies who don't bother to supply a
> valid Host header while they try to log into phpadmin or whatever. For
> non-FANG sites honestly that will be a large proportion of your
> traffic in terms of number of requests. It wasn't making the Racket
> web server sweat, I just wanted to noise out of my logs and stats.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Greg Hendershott]

In my experience a Racket web server will just stubbornly continue to
work for months at a time, if you let it.

(Reminds me of the aviation joke. In the future, cockpits will have
just one human pilot and a dog. The dog is there to bite the human if
they try to turn off the autopilot.)

I have a site on one little t2.micro at AWS.

It is behind an AWS application load balancer -- but mainly just as an
easy way to do SSL termination.

I also added WAF to filter script kiddies who don't bother to supply a
valid Host header while they try to log into phpadmin or whatever. For
non-FANG sites honestly that will be a large proportion of your
traffic in terms of number of requests. It wasn't making the Racket
web server sweat, I just wanted to noise out of my logs and stats.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[George Neuner]

On 11/23/2018 4:38 PM, Philip McGrath wrote:

I'm not familiar with Ruby, so this is just some general information.

The Racket web server already supports concurrency with its built-in 
green threads, so handling one request won't block the concurrent 
handling of another. (Not all languages' built-in web servers do this; 
I don't know about Ruby in particular).


There isn't built-in support for parallelism, i.e. for taking 
advantage of multiple processor cores. This isn't built in, and I 
don't personally need this (my server only has two cores, and it runs 
a database and other processes in addition to Racket). However, it is 
probably possible to recreate the architecture you describe.


Essentially you would be running multiple instances of your 
application behind a load-balancer. If you want to use nginx as a load 
balancer, that's out of the equation; implementing a load-balancer in 
Racket would certainly be possible but probably more work. 
Extra-linguistically, you could just create a few systemd services or 
similar to run several totally separate Racket instances. Within 
Racket, you would want to build on "places." You can have N worker 
places running N instances of your application on N OS-level threads, 
plus a master place to control them. A mechanism for gracefully 
stopping the Racket web server is built in (see serve/launch/wait 
). 
Reloading isn't built in, but I've heard good things about the 
"reloadable" package 
(https://pkgs.racket-lang.org/package/reloadable), which should let 
you implement it if needed.


While I only run one instance of our application, I do use places to 
redirect HTTP to HTTPS.


One caveat is that all of this assumes that, if you are using 
continuations at all, you are using serializable continuations with 
`#lang web-server`. Making places work well with native continuations 
would probably be a lot of work, and it would probably be better to 
build that functionality into the Racket web server than to try to 
fake it as a client.


-Philip



Just a comment:

Multiple (identical) server instances running on the same machine can 
listen for connections on the same network port - an incoming client 
call will be connected to only one of them.  You only need to use 
different ports when services are not interchangeable [i.e. it matters 
which one you are talking to].


Process parallelism, in effect, can give you automatic connection load 
balancing on a single machine.  Using a separate load balancer in front 
technically is overkill for such situations, but it is a good design if 
you want to preserve the option to relocate your services to different 
machines.


George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Racket application servers

[Philip McGrath]

I'm not familiar with Ruby, so this is just some general information.

The Racket web server already supports concurrency with its built-in green
threads, so handling one request won't block the concurrent handling of
another. (Not all languages' built-in web servers do this; I don't know
about Ruby in particular).

There isn't built-in support for parallelism, i.e. for taking advantage of
multiple processor cores. This isn't built in, and I don't personally need
this (my server only has two cores, and it runs a database and other
processes in addition to Racket). However, it is probably possible to
recreate the architecture you describe.

Essentially you would be running multiple instances of your application
behind a load-balancer. If you want to use nginx as a load balancer, that's
out of the equation; implementing a load-balancer in Racket would certainly
be possible but probably more work. Extra-linguistically, you could just
create a few systemd services or similar to run several totally separate
Racket instances. Within Racket, you would want to build on "places." You
can have N worker places running N instances of your application on N
OS-level threads, plus a master place to control them. A mechanism for
gracefully stopping the Racket web server is built in (see serve/launch/wait
).
Reloading isn't built in, but I've heard good things about the "reloadable"
package (https://pkgs.racket-lang.org/package/reloadable), which should let
you implement it if needed.

While I only run one instance of our application, I do use places to
redirect HTTP to HTTPS.

One caveat is that all of this assumes that, if you are using continuations
at all, you are using serializable continuations with `#lang web-server`.
Making places work well with native continuations would probably be a lot
of work, and it would probably be better to build that functionality into
the Racket web server than to try to fake it as a client.

-Philip


On Fri, Nov 23, 2018 at 3:46 PM Brian Adkins  wrote:

> I'm porting a web application from Ruby/Rails to Racket, and I'd like
> something to manage the Racket server processes.
>
> In the Ruby world, I'm currently using Unicorn (
> https://en.wikipedia.org/wiki/Unicorn_(web_server) ) prior to that I used
> Nginx Passenger ( https://en.wikipedia.org/wiki/Phusion_Passenger ), etc.
> Another popular Ruby app server is Puma (
> https://en.wikipedia.org/wiki/Puma_(web_server) )
>
> I'll use nginx as the front end web server, and it will proxy to the
> application server. In a nutshell, for Unicorn, I configure the location of
> the Rails app, configure how many processes I want, and Unicorn will spin
> up that number of Rails processes and handle routing requests from nginx to
> each of the Rails processes in some fashion. If a Rails process exists
> abnormally, it will spin up another one to replace it. To deploy a new
> version of my app, I can send a signal to the Unicorn master process to
> *gracefully* restart all the processes i.e. it waits for the current
> request to finish, and then kills the process and spins up a new one using
> the new version of the app.
>
> Are there similar application servers available for Racket? Alternatively,
> if not, and you have long running applications in Racket, what are you
> using to manage them?
>
> Thanks,
> Brian
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.