Re: Clientbodyinfileonly - POST request is discarded
On Wed, Oct 12, 2016 at 03:34:45PM -0400, yurai wrote:
Hi there,
> >Are you reporting that the content of the client-file upload.txt is not
> >saved on the nginx server that is localhost, in a numbered file below
> >your client_body_temp_path?
>
> Yes. Exactly this. My /tmp/nginx-client-body directory is empty.
Ok, that is unexpected to me.
I've read back over the mail thread, and there seem to be a few things
where it is not clear to me what exactly is happening.
Can you make a test nginx.conf that is very simple, in an attempt to
isolate where things are going wrong?
I use this:
==
events {}
http {
server {
listen 8008;
location = /upload {
client_body_temp_path /tmp/clientb;
client_body_in_file_only on;
proxy_set_header X-FILE $request_body_file;
proxy_pass http://127.0.0.1:8008/upstream;
}
location = /upstream {
return 200 "Look in $http_x_file\n";
}
}
}
==
and when I do
curl -v --data-binary words http://127.0.0.1:8008/upload
I see the POST with Content-Length: 5; I get a response of "Look in " and a
filename, and when I "ls -l" that filename I see that it is 5 bytes long.
I use /tmp/clientb as the client directory above; that directory did
not exist before I reloaded nginx, so nginx will create it with suitable
permissions.
When I then do
curl -v --data-binary @upload.txt http://127.0.0.1:8008/upload
I see the POST with Content-Length: 16; I get a response of "Look in "
and a different filename, and when I "cat" that filename I see the same
16-byte content as was in my original local upload.txt file.
When you do exactly that, do you see anything different?
Note that this is *not* exactly the same as your original case, because
it leaves out many of the config directives. In particular, this *does*
send the initial POST content to the upstream. That's ok; the point of
this is to find out why and where the initial set-up is broken. Other
bits can be added afterwards.
> >There is more than one server involved. Please be very clear which one
> >you are referring to, when you refer to any.
>
> Please notice that in many places I try to be precised as much as possible
> by reffering to s1 and s2. Sorry for confusion.
Actually, I'm wrong there, sorry about that. I had got confused with a
separate mail; your mails were clear about the two server blocks on the
one nginx on localhost.
Thanks,
f
--
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Re: Clientbodyinfileonly - POST request is discarded
Hi, >Are you reporting that the content of the client-file upload.txt is not >saved on the nginx server that is localhost, in a numbered file below >your client_body_temp_path? Yes. Exactly this. My /tmp/nginx-client-body directory is empty. >There is more than one server involved. Please be very clear which one >you are referring to, when you refer to any. Please notice that in many places I try to be precised as much as possible by reffering to s1 and s2. Sorry for confusion. By writing "server" I mean s1. By writing "backend" I mean s2. Both server names comes from configuration file I placed on beginning of discussion. Regards, Dawid Posted at Nginx Forum: https://forum.nginx.org/read.php?2,270063,270222#msg-270222 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
Re: Clientbodyinfileonly - POST request is discarded
On Wed, Oct 12, 2016 at 06:28:47AM -0400, yurai wrote: Hi there, > >"The file should be transferred to the nginx server." > > This is the whole point. > With current configuration when I type curl --data-binary '@upload.txt' > http://localhost/upload file is NOT transffered from client to server at all > - "proxy_pass" is performed and I only get HTTP response 200. There is the client. There is the nginx server, that the client talks to. There is the upstream back-end server, that nginx talks to. Are you reporting that the content of the client-file upload.txt is not saved on the nginx server that is localhost, in a numbered file below your client_body_temp_path? Or are you reporting that the content of the client-file upload.txt is not transferred to the upstream back-end server? There is more than one server involved. Please be very clear which one you are referring to, when you refer to any. > When I change my configuration (by removing whole backend configuration (s2 > block) and all proxy_* directives from s1) and type same command I get HTTP > 405 Not Allowed or HTTP 301 Moved Permanently. > > Let's ignore for the second size of my file in body. Maybe in this moment > the right question is: what should I do to make my above curl command work? It works for me. Perhaps I have a different idea of what "works" mean. f -- Francis Dalyfran...@daoine.org ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
Re: Clientbodyinfileonly - POST request is discarded
Hello, >"The file should be transferred to the nginx server." This is the whole point. With current configuration when I type curl --data-binary '@upload.txt' http://localhost/upload file is NOT transffered from client to server at all - "proxy_pass" is performed and I only get HTTP response 200. When I change my configuration (by removing whole backend configuration (s2 block) and all proxy_* directives from s1) and type same command I get HTTP 405 Not Allowed or HTTP 301 Moved Permanently. Let's ignore for the second size of my file in body. Maybe in this moment the right question is: what should I do to make my above curl command work? Regards, Dawid Posted at Nginx Forum: https://forum.nginx.org/read.php?2,270063,270196#msg-270196 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
Re: Clientbodyinfileonly - POST request is discarded
On Mon, Oct 10, 2016 at 03:41:13AM -0400, yurai wrote:
Hi there,
> thank you for response. I just want to transfer big file on Nginx server
> inside POST request. I use method from:
> https://coderwall.com/p/swgfvw/nginx-direct-file-upload-without-passing-them-through-backend
>
> Whole my analysis and expectations are based on this article.
I think that at least one of us is confused.
You have "client" - "nginx" - "backend".
That document is about getting a file from "client" to "nginx", and then
telling "backend" what filename is used on "nginx".
If "backend" wants to access the file, that is out of the scope of that
document. ("backend" gets the filename, and should presumably do a
separate "open" on the shared filesystem, or have a separate transfer
to be able to read the file.)
> Unfotunately this "clientbodyinfileonly" functionality is not well
> documented so I'm not sure how exactly ok scenario from Nginx POV should
> look like. I just know that my file is not transfered and not saved on
> server side.
The file should be transferred to the nginx server.
If that happens, the nginx side is doing what it was configured to do.
The functionality is documented at
http://nginx.org/r/client_body_in_file_only
Cheers,
f
--
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Re: Clientbodyinfileonly - POST request is discarded
Hello Francis, thank you for response. I just want to transfer big file on Nginx server inside POST request. I use method from: https://coderwall.com/p/swgfvw/nginx-direct-file-upload-without-passing-them-through-backend Whole my analysis and expectations are based on this article. Unfotunately this "clientbodyinfileonly" functionality is not well documented so I'm not sure how exactly ok scenario from Nginx POV should look like. I just know that my file is not transfered and not saved on server side. Regards, Dawid Posted at Nginx Forum: https://forum.nginx.org/read.php?2,270063,270145#msg-270145 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
Re: Clientbodyinfileonly - POST request is discarded
Hi Francis,
I added return statement to my config as you suggested. Now config for
backend s2 looks like:
server {
listen 8080;
server_name s2;
location / {
root /usr/share/nginx/html/foo/bar;
return 200 "Do something sensible with $http_x_file\n";
autoindex on;
}
}
Unfortunately still it doesn't work as I expect - upload.txt file content is
not saved on server side in /tmp/nginx-client-body.
My understanding is that transfer should be performed in 2 phases (2 POST
requests via 2x curl). First request should deliver file name, and second
should deliver actual file content without ingeration from backend side. I
analyzed HTTP flow in wireshark and it looks fine for me (details below).
1. curl --data-binary upload.txt http://localhost/upload
- s1 listening on 80 recive POST request with body = "upload.txt". S1 buffer
"upload.txt" in /tmp/001, generate new POST request with field
X-FILE="/tmp/001" and pass this request to backend (s2)
- s2 listening on 8080 recieve POST request with X-FILE = "/tmp/001"
- s2 generate HTTP response 200 with body = "Do something sensible with
/tmp/001\n" and pass it to s1
- s1 recieve above response and pass it to client
- client recieve HTTP response 200 with body = "Do something sensible with
/tmp/001\n"
2. curl --data-binary '@upload.txt' http://localhost/upload
If I understand this mechanism correctly now actual upload.txt transfer to
server without backend ingeration should be triggered.
So I should get reponse 200 and upload.txt content should be saved by server
under /tmp/nginx-client-body.
Anyway when I type curl --data-binary '@upload.txt' http://localhost/upload
whole scenario from previous point is performed again.
Posted at Nginx Forum:
https://forum.nginx.org/read.php?2,270063,270122#msg-270122
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Re: Clientbodyinfileonly - POST request is discarded
On Tue, Oct 04, 2016 at 03:28:18PM -0400, yurai wrote:
Hi there,
> Unfortunately I get "HTTP/1.1 405 Not Allowed" error code all the time.
The "back-end" thing that you POST to must be able to handle the POST.
Right now, you just ask nginx to serve a file from the filesystem,
which does not accept a POST (by default).
> location / {
> root /usr/share/nginx/html/foo/bar;
> autoindex on;
> }
Add something like
return 200 "Do something sensible with $http_x_file\n";
in there and you'll see that it does work.
And then decide what you actually want to do with the file, and make
something do that.
Good luck with it,
f
--
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
