Re: [ceph-users] nginx (tengine) and radosgw
Hello, Sylvain Could you please share your civetweb conf for radosgw? If launched directly (via --rgw-frontends civetweb port=80) it has issues with Horizon, maybe I'm missing something. 2014-07-07 14:41 GMT+03:00 Sylvain Munaut s.mun...@whatever-company.com: Hi, if anyone else is looking to run radosgw without having to run apache, I would recommend you look into tengine :) Just as a side note, you can now try the civetweb backend of rgw so you don't need fastcgi at all. We started running it that way and so far, it's working pretty good. Cheers, Sylvain ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] nginx (tengine) and radosgw
Hi guys.
Has anyone managed to get radosgw+tengine work with horizon+nginx?
I'm experiencing problems with this installation, more info here:
http://www.mail-archive.com/openstack@lists.openstack.org/msg07934.html
2014-07-04 15:05 GMT+03:00 Andrei Mikhailovsky and...@arhont.com:
Hi David,
Do you mind sharing the howto/documentation with examples of configs, etc.?
I am tempted to give it a go and replace the Apache reverse proxy that I
am currently using.
cheers
Andrei
--
*From: *David Moreau Simard dmsim...@iweb.com
*To: *ceph-users@lists.ceph.com
*Sent: *Sunday, 22 June, 2014 2:37:00 AM
*Subject: *Re: [ceph-users] nginx (tengine) and radosgw
Hi,
I just wanted to chime in and say that I didn’t notice any problems
swapping nginx out in favor of tengine.
tengine is used as a load balancer that also handles SSL termination.
I found that disabling body buffering saves a lot on upload times as
well.
I took the time to do a post about it and linked this thread:
http://dmsimard.com/2014/06/21/a-use-case-of-tengine-a-drop-in-replacement-and-fork-of-nginx/
- David
On May 29, 2014, at 12:20 PM, Michael Lukzak mis...@vp.pl wrote:
Hi,
Ups, so I don't read carefully a doc...
I will try this solution.
Thanks!
Michael
From the docs, you need this setting in ceph.conf (if you're using
nginx/tengine):
rgw print continue = false
This will fix the 100-continue issues.
On 5/29/2014 5:56 AM, Michael Lukzak wrote:
Re[2]: [ceph-users] nginx (tengine) and radosgw Hi,
I'm also use tengine, works fine with SSL (I have a Wildcard).
But I have other issue with HTTP 100-Continue.
Clients like boto or Cyberduck hangs if they can't make HTTP 100-Continue.
IP_REMOVED - - [29/May/2014:11:27:53 +] PUT
/temp/1b6f6a11d7aa188f06f8255fdf0345b4 HTTP/1.1 100 0 - Boto/2.27.0
Python/2.7.6 Linux/3.13.0-24-generic
Do You have also problem with that?
I used for testing oryginal nginx and also have a problem with
100-Continue.
Only Apache 2.x works fine.
BR,
Michael
I haven't tried SSL yet. We currently don't have a wildcard certificate
for this, so it hasn't been a concern (and our current use case, all the
files are public anyway).
On 5/20/2014 4:26 PM, Andrei Mikhailovsky wrote:
That looks very interesting indeed. I've tried to use nginx, but from what
I recall it had some ssl related issues. Have you tried to make the ssl
work so that nginx acts as an ssl proxy in front of the radosgw?
Cheers
Andrei
--
*From: *Brian Rak b...@gameservers.com b...@gameservers.com
*To: *ceph-users@lists.ceph.com
*Sent: *Tuesday, 20 May, 2014 9:11:58 PM
*Subject: *[ceph-users] nginx (tengine) and radosgw
I've just finished converting from nginx/radosgw to tengine/radosgw, and
it's fixed all the weird issues I was seeing (uploads failing, random
clock skew errors, timeouts).
The problem with nginx and radosgw is that nginx insists on buffering
all the uploads to disk. This causes a significant performance hit, and
prevents larger uploads from working. Supposedly, there is going to be
an option in nginx to disable this, but it hasn't been released yet (nor
do I see anything on the nginx devel list about it).
tengine ( http://tengine.taobao.org/ ) is an nginx fork that implements
unbuffered uploads to fastcgi. It's basically a drop in replacement for
nginx.
My configuration looks like this:
server {
listen 80;
server_name *.rados.test rados.test;
client_max_body_size 10g;
# This is the important option that tengine has, but nginx does not
fastcgi_request_buffering off;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
if ($request_method = PUT ) {
rewrite ^ /PUT$request_uri;
}
include fastcgi_params;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
location /PUT/ {
internal;
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
include fastcgi_params;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
}
if anyone else is looking to run radosgw without having to run apache, I
would recommend you look into tengine :)
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] nginx (tengine) and radosgw
Hi, if anyone else is looking to run radosgw without having to run apache, I would recommend you look into tengine :) Just as a side note, you can now try the civetweb backend of rgw so you don't need fastcgi at all. We started running it that way and so far, it's working pretty good. Cheers, Sylvain ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] nginx (tengine) and radosgw
Hi David,
Do you mind sharing the howto/documentation with examples of configs, etc.?
I am tempted to give it a go and replace the Apache reverse proxy that I am
currently using.
cheers
Andrei
- Original Message -
From: David Moreau Simard dmsim...@iweb.com
To: ceph-users@lists.ceph.com
Sent: Sunday, 22 June, 2014 2:37:00 AM
Subject: Re: [ceph-users] nginx (tengine) and radosgw
Hi,
I just wanted to chime in and say that I didn’t notice any problems swapping
nginx out in favor of tengine.
tengine is used as a load balancer that also handles SSL termination.
I found that disabling body buffering saves a lot on upload times as well.
I took the time to do a post about it and linked this thread:
http://dmsimard.com/2014/06/21/a-use-case-of-tengine-a-drop-in-replacement-and-fork-of-nginx/
- David
On May 29, 2014, at 12:20 PM, Michael Lukzak mis...@vp.pl wrote:
Re[2]: [ceph-users] nginx (tengine) and radosgw
Hi,
Ups, so I don't read carefully a doc...
I will try this solution.
Thanks!
Michael
From the docs, you need this setting in ceph.conf (if you're using
nginx/tengine):
rgw print continue = false
This will fix the 100-continue issues.
On 5/29/2014 5:56 AM, Michael Lukzak wrote:
Re[2]: [ceph-users] nginx (tengine) and radosgw Hi,
I'm also use tengine, works fine with SSL (I have a Wildcard).
But I have other issue with HTTP 100-Continue.
Clients like boto or Cyberduck hangs if they can't make HTTP 100-Continue.
IP_REMOVED - - [29/May/2014:11:27:53 +] PUT
/temp/1b6f6a11d7aa188f06f8255fdf0345b4 HTTP/1.1 100 0 - Boto/2.27.0
Python/2.7.6 Linux/3.13.0-24-generic
Do You have also problem with that?
I used for testing oryginal nginx and also have a problem with 100-Continue.
Only Apache 2.x works fine.
BR,
Michael
I haven't tried SSL yet. We currently don't have a wildcard certificate
for this, so it hasn't been a concern (and our current use case, all the files
are public anyway).
On 5/20/2014 4:26 PM, Andrei Mikhailovsky wrote:
That looks very interesting indeed. I've tried to use nginx, but from what I
recall it had some ssl related issues. Have you tried to make the ssl work so
that nginx acts as an ssl proxy in front of the radosgw?
Cheers
Andrei
From: Brian Rak b...@gameservers.com
To: ceph-users@lists.ceph.com
Sent: Tuesday, 20 May, 2014 9:11:58 PM
Subject: [ceph-users] nginx (tengine) and radosgw
I've just finished converting from nginx/radosgw to tengine/radosgw, and
it's fixed all the weird issues I was seeing (uploads failing, random
clock skew errors, timeouts).
The problem with nginx and radosgw is that nginx insists on buffering
all the uploads to disk. This causes a significant performance hit, and
prevents larger uploads from working. Supposedly, there is going to be
an option in nginx to disable this, but it hasn't been released yet (nor
do I see anything on the nginx devel list about it).
tengine ( http://tengine.taobao.org/ ) is an nginx fork that implements
unbuffered uploads to fastcgi. It's basically a drop in replacement for
nginx.
My configuration looks like this:
server {
listen 80;
server_name *.rados.test rados.test;
client_max_body_size 10g;
# This is the important option that tengine has, but nginx does not
fastcgi_request_buffering off;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
if ($request_method = PUT ) {
rewrite ^ /PUT$request_uri;
}
include fastcgi_params;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
location /PUT/ {
internal;
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
include fastcgi_params;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
}
if anyone else is looking to run radosgw without having to run apache, I
would recommend you look into tengine :)
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] nginx (tengine) and radosgw
From the docs, you need this setting in ceph.conf (if you're using
nginx/tengine):
rgw print continue = false
This will fix the 100-continue issues.
On 5/29/2014 5:56 AM, Michael Lukzak wrote:
Re[2]: [ceph-users] nginx (tengine) and radosgw Hi,
I'm also use tengine, works fine with SSL (I have a Wildcard).
But I have other issue with HTTP 100-Continue.
Clients like boto or Cyberduck hangs if they can't make HTTP 100-Continue.
IP_REMOVED - - [29/May/2014:11:27:53 +] PUT
/temp/1b6f6a11d7aa188f06f8255fdf0345b4 HTTP/1.1 100 0 -
Boto/2.27.0 Python/2.7.6 Linux/3.13.0-24-generic
Do You have also problem with that?
I used for testing oryginal nginx and also have a problem with
100-Continue.
Only Apache 2.x works fine.
BR,
Michael
I haven't tried SSL yet. We currently don't have a wildcard
certificate for this, so it hasn't been a concern (and our current use
case, all the files are public anyway).
On 5/20/2014 4:26 PM, Andrei Mikhailovsky wrote:
That looks very interesting indeed. I've tried to use nginx, but from
what I recall it had some ssl related issues. Have you tried to make
the ssl work so that nginx acts as an ssl proxy in front of the radosgw?
Cheers
Andrei
*From: *Brian Rak b...@gameservers.com mailto:b...@gameservers.com
*To: *ceph-users@lists.ceph.com mailto:ceph-users@lists.ceph.com
*Sent: *Tuesday, 20 May, 2014 9:11:58 PM
*Subject: *[ceph-users] nginx (tengine) and radosgw
I've just finished converting from nginx/radosgw to tengine/radosgw, and
it's fixed all the weird issues I was seeing (uploads failing, random
clock skew errors, timeouts).
The problem with nginx and radosgw is that nginx insists on buffering
all the uploads to disk. This causes a significant performance hit, and
prevents larger uploads from working. Supposedly, there is going to be
an option in nginx to disable this, but it hasn't been released yet (nor
do I see anything on the nginx devel list about it).
tengine ( http://tengine.taobao.org/) is an nginx fork that implements
unbuffered uploads to fastcgi. It's basically a drop in replacement for
nginx.
My configuration looks like this:
server {
listen 80;
server_name *.rados.test rados.test;
client_max_body_size 10g;
# This is the important option that tengine has, but nginx
does not
fastcgi_request_buffering off;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
if ($request_method = PUT ) {
rewrite ^ /PUT$request_uri;
}
include fastcgi_params;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
location /PUT/ {
internal;
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
include fastcgi_params;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
}
if anyone else is looking to run radosgw without having to run apache, I
would recommend you look into tengine :)
___
ceph-users mailing list
ceph-users@lists.ceph.com mailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] nginx (tengine) and radosgw
Hi,
Ups, so I don't read carefully a doc...
I will try this solution.
Thanks!
Michael
From the docs, you need this setting in ceph.conf (if you're using
nginx/tengine):
rgw print continue = false
This will fix the 100-continue issues.
On 5/29/2014 5:56 AM, Michael Lukzak wrote:
Re[2]: [ceph-users] nginx (tengine) and radosgw Hi,
I'm also use tengine, works fine with SSL (I have a Wildcard).
But I have other issue with HTTP 100-Continue.
Clients like boto or Cyberduck hangs if they can't make HTTP 100-Continue.
IP_REMOVED - - [29/May/2014:11:27:53 +] PUT
/temp/1b6f6a11d7aa188f06f8255fdf0345b4 HTTP/1.1 100 0 - Boto/2.27.0
Python/2.7.6 Linux/3.13.0-24-generic
Do You have also problem with that?
I used for testing oryginal nginx and also have a problem with 100-Continue.
Only Apache 2.x works fine.
BR,
Michael
I haven't tried SSL yet. We currently don't have a wildcard certificate for
this, so it hasn't been a concern (and our current use case, all the files are
public anyway).
On 5/20/2014 4:26 PM, Andrei Mikhailovsky wrote:
That looks very interesting indeed. I've tried to use nginx, but from what I
recall it had some ssl related issues. Have you tried to make the ssl work so
that nginx acts as an ssl proxy in front of the radosgw?
Cheers
Andrei
From: Brian Rak b...@gameservers.com
To: ceph-users@lists.ceph.com
Sent: Tuesday, 20 May, 2014 9:11:58 PM
Subject: [ceph-users] nginx (tengine) and radosgw
I've just finished converting from nginx/radosgw to tengine/radosgw, and
it's fixed all the weird issues I was seeing (uploads failing, random
clock skew errors, timeouts).
The problem with nginx and radosgw is that nginx insists on buffering
all the uploads to disk. This causes a significant performance hit, and
prevents larger uploads from working. Supposedly, there is going to be
an option in nginx to disable this, but it hasn't been released yet (nor
do I see anything on the nginx devel list about it).
tengine ( http://tengine.taobao.org/ ) is an nginx fork that implements
unbuffered uploads to fastcgi. It's basically a drop in replacement for
nginx.
My configuration looks like this:
server {
listen 80;
server_name *.rados.test rados.test;
client_max_body_size 10g;
# This is the important option that tengine has, but nginx does not
fastcgi_request_buffering off;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
if ($request_method = PUT ) {
rewrite ^ /PUT$request_uri;
}
include fastcgi_params;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
location /PUT/ {
internal;
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
include fastcgi_params;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
}
if anyone else is looking to run radosgw without having to run apache, I
would recommend you look into tengine :)
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] nginx (tengine) and radosgw
That looks very interesting indeed. I've tried to use nginx, but from what I
recall it had some ssl related issues. Have you tried to make the ssl work so
that nginx acts as an ssl proxy in front of the radosgw?
Cheers
Andrei
- Original Message -
From: Brian Rak b...@gameservers.com
To: ceph-users@lists.ceph.com
Sent: Tuesday, 20 May, 2014 9:11:58 PM
Subject: [ceph-users] nginx (tengine) and radosgw
I've just finished converting from nginx/radosgw to tengine/radosgw, and
it's fixed all the weird issues I was seeing (uploads failing, random
clock skew errors, timeouts).
The problem with nginx and radosgw is that nginx insists on buffering
all the uploads to disk. This causes a significant performance hit, and
prevents larger uploads from working. Supposedly, there is going to be
an option in nginx to disable this, but it hasn't been released yet (nor
do I see anything on the nginx devel list about it).
tengine ( http://tengine.taobao.org/ ) is an nginx fork that implements
unbuffered uploads to fastcgi. It's basically a drop in replacement for
nginx.
My configuration looks like this:
server {
listen 80;
server_name *.rados.test rados.test;
client_max_body_size 10g;
# This is the important option that tengine has, but nginx does not
fastcgi_request_buffering off;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
if ($request_method = PUT ) {
rewrite ^ /PUT$request_uri;
}
include fastcgi_params;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
location /PUT/ {
internal;
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
include fastcgi_params;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
}
if anyone else is looking to run radosgw without having to run apache, I
would recommend you look into tengine :)
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] nginx (tengine) and radosgw
I haven't tried SSL yet. We currently don't have a wildcard certificate
for this, so it hasn't been a concern (and our current use case, all the
files are public anyway).
On 5/20/2014 4:26 PM, Andrei Mikhailovsky wrote:
That looks very interesting indeed. I've tried to use nginx, but from
what I recall it had some ssl related issues. Have you tried to make
the ssl work so that nginx acts as an ssl proxy in front of the radosgw?
Cheers
Andrei
*From: *Brian Rak b...@gameservers.com
*To: *ceph-users@lists.ceph.com
*Sent: *Tuesday, 20 May, 2014 9:11:58 PM
*Subject: *[ceph-users] nginx (tengine) and radosgw
I've just finished converting from nginx/radosgw to tengine/radosgw, and
it's fixed all the weird issues I was seeing (uploads failing, random
clock skew errors, timeouts).
The problem with nginx and radosgw is that nginx insists on buffering
all the uploads to disk. This causes a significant performance hit, and
prevents larger uploads from working. Supposedly, there is going to be
an option in nginx to disable this, but it hasn't been released yet (nor
do I see anything on the nginx devel list about it).
tengine ( http://tengine.taobao.org/ ) is an nginx fork that implements
unbuffered uploads to fastcgi. It's basically a drop in replacement for
nginx.
My configuration looks like this:
server {
listen 80;
server_name *.rados.test rados.test;
client_max_body_size 10g;
# This is the important option that tengine has, but nginx
does not
fastcgi_request_buffering off;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
if ($request_method = PUT ) {
rewrite ^ /PUT$request_uri;
}
include fastcgi_params;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
location /PUT/ {
internal;
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
include fastcgi_params;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass unix:/path/to/ceph.radosgw.fastcgi.sock;
}
}
if anyone else is looking to run radosgw without having to run apache, I
would recommend you look into tengine :)
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
