RE: Help on designing using multiple location/upstream modules
Hi! Date: Sat, 30 Nov 2013 14:37:04 -0800 Subject: Re: Help on designing using multiple location/upstream modules From: agen...@gmail.com To: nginx-devel@nginx.org Hello! On Sat, Nov 30, 2013 at 11:05 AM, Reetesh Ranjan wrote: Thanks for the help! Going by the documentation of the lua-nginx-module on its subrequest handling it looks really promising for my use case. Great :) I am currently writing the Sphinx2 upstream module. BTW, you could also build a lua-resty-sphinx2 library atop ngx_lua's nonblocking cosocket API instead of writing an upstream C module. You can check out the lua-resty-mysql or lua-resty-redis libraries for examples: https://github.com/agentzh/lua-resty-mysql https://github.com/agentzh/lua-resty-redis The ngx_lua cosocket API is more flexible and much easier to use :) That way you also don't have to use nginx subrequests at all ;) I was already midway writing the upstream module, and completed it. It was a great exercise anyway for looking into guts of nginx upstream state machine. :) I would look at the lua ones next. I am sure the learning obtained here and the eventual approach I had to take would be useful to build lua-resty-sphinx2. Looking forward to be part of the great openresty bundle! :) Would get back with questions in case I have any on using the lua-nginx-module for achieving what I need to do. You're recommended to post ngx_lua related questions to the openresty-en mailing list: https://groups.google.com/group/openresty-en That way we can see your mails sooner rather later :) Best regards, -agentzh ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
RE: Help on designing using multiple location/upstream modules
Hi Yichun Zhang, Thanks for the help! Going by the documentation of the lua-nginx-module on its subrequest handling it looks really promising for my use case. I am currently writing the Sphinx2 upstream module. Would get back with questions in case I have any on using the lua-nginx-module for achieving what I need to do. Regards,Reetesh Date: Wed, 27 Nov 2013 12:39:15 -0800 Subject: Re: Help on designing using multiple location/upstream modules From: agen...@gmail.com To: nginx-devel@nginx.org Hello! On Tue, Nov 26, 2013 at 4:30 AM, Reetesh Ranjan wrote: I have thought of the following design, in terms of nginx modules I would use: 1 My main location module that picks the keywords entered and communicates to Redis and Sphinx 2 For communicating to Redis I thought of using HttpRedis2Module (http://wiki.nginx.org/HttpRedis2Module) 3 For communicating with Sphinx, I am trying to write a simple C++ client or adapt the Sphinx C++ client (http://sourceforge.net/projects/cppsphinxclient/) or its parts into an upstream module. This looks trivial if you use ngx_lua module as the glue. In particular you can check out the ngx.location.capture and ngx.location.capture_multi API functions for captured subrequests: https://github.com/chaoslawful/lua-nginx-module#ngxlocationcapture https://github.com/chaoslawful/lua-nginx-module#ngxlocationcapture_multi And probably also the light thread API that can work with the subrequest API above: https://github.com/chaoslawful/lua-nginx-module#ngxthreadspawn When using the Lua API provided by ngx_lua, everything is nonblocking out of the box :) Regards, -agentzh ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Help on designing using multiple location/upstream modules
Hello! On Tue, Nov 26, 2013 at 4:30 AM, Reetesh Ranjan wrote: I have thought of the following design, in terms of nginx modules I would use: 1 My main location module that picks the keywords entered and communicates to Redis and Sphinx 2 For communicating to Redis I thought of using HttpRedis2Module (http://wiki.nginx.org/HttpRedis2Module) 3 For communicating with Sphinx, I am trying to write a simple C++ client or adapt the Sphinx C++ client (http://sourceforge.net/projects/cppsphinxclient/) or its parts into an upstream module. This looks trivial if you use ngx_lua module as the glue. In particular you can check out the ngx.location.capture and ngx.location.capture_multi API functions for captured subrequests: https://github.com/chaoslawful/lua-nginx-module#ngxlocationcapture https://github.com/chaoslawful/lua-nginx-module#ngxlocationcapture_multi And probably also the light thread API that can work with the subrequest API above: https://github.com/chaoslawful/lua-nginx-module#ngxthreadspawn When using the Lua API provided by ngx_lua, everything is nonblocking out of the box :) Regards, -agentzh ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
RE: Help on designing using multiple location/upstream modules
I saw several pages on web about ngx_http_subrequest; filters vs location modules using it; parallel vs sequential usage; code/modules using it etc. Would first try out a solution for my use case using this method, and come back in case I am stuck and of course after reading previous threads. Regards,Reetesh From: reete...@outlook.com To: nginx-devel@nginx.org Subject: Help on designing using multiple location/upstream modules Date: Tue, 26 Nov 2013 18:00:35 +0530 Hi, I am a newbie to nginx. I have done some initial research on nginx architecture, location modules, upstream modules, third party modules available for various purposes etc. After going through a number of pages I have a question which I can't seem to find an easy answer to. I have a very simple use case like this: user enters a set of keywords to search on my web site. In the backend, in my nginx location module, i first go to Redis for cached results against the set of keywords and if not found, to Sphinx search daemon. In the latter case, I set the results obtained from Sphinx back into Redis. I have thought of the following design, in terms of nginx modules I would use: 1 My main location module that picks the keywords entered and communicates to Redis and Sphinx2 For communicating to Redis I thought of using HttpRedis2Module (http://wiki.nginx.org/HttpRedis2Module)3 For communicating with Sphinx, I am trying to write a simple C++ client or adapt the Sphinx C++ client (http://sourceforge.net/projects/cppsphinxclient/) or its parts into an upstream module. What I wanted to know is how to invoke the upstream modules within my main location module. Are there standard APIs provided by Nginx for the same and do they retain the async advantages? Or do I have to resort to make curl calls from my C++ client and use the response? I was hoping that the former (Nginx APIs to call upstream modules) exists in some form and serves as some shortcut or faster way/alternative to making some curl API calls. Regards,Reetesh ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
