Re: Caching servers in Local ISPs !!

2014-06-18 Thread shahzaib shahzaib
ok, but i have no idea why ISP is asking for BGP and matter of fact is, i'll have to make BGP work somehow, so local caching server will fetch the new subnets from ISP router automatically (and i don't know how). Btw, our local ISP provided us with some testing ip prefixes to check nginx based

Re: Caching servers in Local ISPs !!

2014-06-18 Thread itpp2012
shahzaib1232 Wrote: --- Btw, our local ISP provided us with some testing ip prefixes to check nginx based caching. i.e geo { default 0; 10.0.0.0/8 39.23.2.0/24 1; 112.50.192.0/18 1; } Typo?? geo { default 0; 10.0.0.0/8 1;

Re: Caching servers in Local ISPs !!

2014-06-18 Thread shahzaib shahzaib
geo { default 0; 10.0.0.0/8 1; 39.23.2.0/24 1; 112.50.192.0/18 1; } Sorry i didn't write accurately here but it is 10.0.0.0/8 1; in nginx config, so the problem is not the wrong syntax for geo {}. On Wed, Jun 18, 2014 at 11:24 PM, itpp2012 nginx-fo...@nginx.us wrote: shahzaib1232 Wrote:

RE: Caching servers in Local ISPs !!

2014-06-18 Thread Lukas Tribus
Hi, ok, but i have no idea why ISP is asking for BGP and matter of fact is, i'll have to make BGP work somehow, so local caching server will fetch the new subnets from ISP router automatically (and i don't know how). I strongly suggest you hire some consultant who can help you setting all

Re: Caching servers in Local ISPs !!

2014-06-18 Thread shahzaib shahzaib
why not host those file on a professional CDN instead of in-house? Because 80% of the traffic is from our country and 50% of that traffic is from the ISP we're talking to and this is the reason we deployed the caching box on this ISP edge. On Thu, Jun 19, 2014 at 12:35 AM, Lukas Tribus

Re: Caching servers in Local ISPs !!

2014-06-18 Thread Jonathan Matthews
On 18 Jun 2014 20:45, shahzaib shahzaib shahzaib...@gmail.com wrote: why not host those file on a professional CDN instead of in-house? Because 80% of the traffic is from our country and 50% of that traffic is from the ISP we're talking to and this is the reason we deployed the caching box on

Re: Caching servers in Local ISPs !!

2014-06-18 Thread shahzaib shahzaib
@Jonathon, yes you're right i should not post off-topic here, offcourse i thought as nginx has tremendous amount of capabilities and there might be alternative possibility of BGP too but i was wrong. I would be thankful if you help me on ngx-http_geo_module as it is related to nginx and help me

Re: Caching servers in Local ISPs !!

2014-06-17 Thread itpp2012
shahzaib1232 Wrote: --- Thanks itpp but the issue is still same and still the ip is from the main server in inspect element as well as in local-caching nginx access logs, i am getting the client ip as main-server's ip instead of original

Re: Caching servers in Local ISPs !!

2014-06-17 Thread itpp2012
shahzaib1232 Wrote: --- i don't think the solution rdns will be suitable for us. I have checked the zebra software to make linux a BGP router http://www.techrepublic.com/article/use-zebra-to-set-up-a-linux-bgp-os pf-router/ Could you tell

Re: Caching servers in Local ISPs !!

2014-06-17 Thread shahzaib shahzaib
Why not use a DNS for the clients? How i would be sure that request coming from the ISP-1 on the DNS server and then point it to the local caching server? I mean i can use View directive of BIND to route specific ips (local ISP clients) to the local caching server and what if tomorrow the ISP has

Re: Caching servers in Local ISPs !!

2014-06-17 Thread shahzaib shahzaib
Why not use a DNS for the clients? How i would be sure that request coming from the ISP-1 on the DNS server ? and then point it to the local caching server? I mean i can use View directive of BIND to route specific ips (local ISP clients) to the local caching server and what if tomorrow the ISP

Re: Caching servers in Local ISPs !!

2014-06-17 Thread itpp2012
You don't need to do anything with a dns that is only local to the clients served by the ISP. Suppose I am in Africa; Question to my ISP: I'd like to go to new-york ISP: new-york is located in south-Africa Suppose I am in the US; Question to my ISP: I'd like to go to new-york ISP: new-york is

Re: Caching servers in Local ISPs !!

2014-06-16 Thread shahzaib shahzaib
Hello itpp,i have been abled to use ngx_http_geo_module. Now the request coming from local ISP will first go to the main server (US) and then main server will check if the ip is 1.2.3.4 so it'll direct the request to the local caching server and than caching server will check if the file is

Re: Caching servers in Local ISPs !!

2014-06-16 Thread itpp2012
shahzaib1232 Wrote: --- Maybe i need to add some variable to get original server ip ? https://www.google.nl/#q=nginx+geo+remote+ip+address http://www.cyberciti.biz/faq/nginx-redirect-backend-traffic-based-upon-client-ip-address/ Posted at

Re: Caching servers in Local ISPs !!

2014-06-06 Thread shahzaib shahzaib
@itpp I am currenlty proceeding with proxy_cache method just because i had to done this in emergency mode due to boss pressure :-|. I have a quick question, can i make nginx to cache files for specific clients ? Like, if our caching servers are deployed by only single ISP named ptcl. So if ip

Re: Caching servers in Local ISPs !!

2014-06-06 Thread itpp2012
shahzaib1232 Wrote: --- @itpp I am currenlty proceeding with proxy_cache method just because i had to done this in emergency mode due to boss pressure :-|. I have a quick question, can i make nginx to cache files for specific clients ?

Re: Caching servers in Local ISPs !!

2014-06-06 Thread shahzaib shahzaib
Thanks a lot itpp. :) I'll look into it and get back to you. Thanks again for quick solution :) On Fri, Jun 6, 2014 at 8:26 PM, itpp2012 nginx-fo...@nginx.us wrote: shahzaib1232 Wrote: --- @itpp I am currenlty proceeding with proxy_cache

Re: Caching servers in Local ISPs !!

2014-06-04 Thread shahzaib shahzaib
@itpp, i just used your method try_files and it worked flawlessly :). Following is the testing config : server { listen 80; server_name domain.com; root /var/www/html/files; location / { root /var/www/html/files; try_files $uri @getfrom_origin; }

Re: Caching servers in Local ISPs !!

2014-06-04 Thread shahzaib shahzaib
Also sync to a temp folder and move after completion or nginx will attempt to send partial files. Oh right. Thanks for quick help and suggestion :). I'll look into wanproxy now. On Thu, Jun 5, 2014 at 12:19 AM, itpp2012 nginx-fo...@nginx.us wrote: shahzaib1232 Wrote:

Re: Caching servers in Local ISPs !!

2014-05-28 Thread itpp2012
shahzaib1232 Wrote: --- Thanks itpp. So according to you the more preferable way is non-caching replication of main content servers to local ISP servers and than run rsync periodically for replication. So if file not found in local ISP

Re: Caching servers in Local ISPs !!

2014-05-28 Thread shahzaib shahzaib
Right. I'll proceed with my research and get back to you with better approach . :) ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx

Re: Caching servers in Local ISPs !!

2014-05-27 Thread shahzaib shahzaib
Hello , Came after a long time to this thread. I have finished the caching method. I've used bind-geoip to resolve all requests from asia against the local ISP caching server ips. Now, here's the most critical part. Suppose a request came from asia for the video file which is not yet

Re: Caching servers in Local ISPs !!

2014-05-27 Thread itpp2012
I'd call that provisioning, a simple try_files directive can deal with that while the provisioning service syncs. http://nginx.org/en/docs/http/ngx_http_core_module.html#try_files Then jump left for cached contents or jump right for non-cached contents. Posted at Nginx Forum:

Re: Caching servers in Local ISPs !!

2014-05-27 Thread shahzaib shahzaib
@itpp, thanks for reply. I am working on proxy_cache . My all webservers in local ISP will be proxy servers and all requests regarding flv,mp4,jpeg from asia will be route through the proxy servers and data will be cache in proxy servers. I think, this will help me with caching. I am testing it on

Re: Caching servers in Local ISPs !!

2014-05-27 Thread itpp2012
I'm not seeing where you determine if a file is present on an edge system, the try_files directive allows you to redirect or not (locally present or not). I don't really see a point for using (proxy)cache, the file is present or it is not. location / { try_files $uri @getfrom_origin; } location

Re: Caching servers in Local ISPs !!

2014-05-27 Thread shahzaib shahzaib
Hello itpp, the Local caching server is nothing but a proxy server which will route all traffic from asia to origin server and will cache the requested file in /data/nginx/cache folder. So when the request made again, it'll be served from cache. I'm not seeing where you determine if a

Re: Caching servers in Local ISPs !!

2014-05-27 Thread itpp2012
Of course this will work but when nginx restarts or the cache gets damaged/corrupted the whole shebang starts again And you can't control the cached files versus try_files and a normal filesystem. Posted at Nginx Forum: http://forum.nginx.org/read.php?2,249997,250395#msg-250395

Re: Caching servers in Local ISPs !!

2014-05-10 Thread shahzaib shahzaib
Thanks for replying guyz. Can i use nginx (origin and edge) ? As the question in following link. http://stackoverflow.com/questions/10024981/distributed-cached-mp4-pseudostreaming-seeking-with-nginx If i use the origin and edge method, i think i'll change my application codes to redirect local

Re: Caching servers in Local ISPs !!

2014-05-10 Thread itpp2012
See http://en.wikipedia.org/wiki/Content_delivery_network and http://en.wikipedia.org/wiki/File:Akamaiprocess.png Make yourself a HLD (high level design) before getting to technology. Posted at Nginx Forum: http://forum.nginx.org/read.php?2,249997,250007#msg-250007

Caching servers in Local ISPs !!

2014-05-09 Thread shahzaib shahzaib
Hello, We're running a high traffic website similar to youtube.com. Due to high bandwidth utilization over the network, we're in contact with the local ISP in order to put caching server to reduce bandwidth utilization for file streaming. Our main front end content servers (nginx) are

Re: Caching servers in Local ISPs !!

2014-05-09 Thread shahzaib shahzaib
@Rainer, we're already in contact with one of our Country's ISP(80% of the country users are using that ISP services) .So, they can do much better work than the DNS sites you provided because we only required caching for our country. On Fri, May 9, 2014 at 10:18 PM, Rainer Duffner

Re: Caching servers in Local ISPs !!

2014-05-09 Thread itpp2012
So on easy note, i would have to assign those machines the preferred dns and use rsync on regular basis in order to make identical data between local caching machines and main front end content servers ? Yep. What if a client request a video which is not in local caching server ? You

Re: Caching servers in Local ISPs !!

2014-05-09 Thread Steve Holdoway
You might want to look at lsyncd - a GZSOC project - to ease the synchronisation. I have had good results with it. Steve On Sat, 2014-05-10 at 00:22 +0500, shahzaib shahzaib wrote: @itpp thanks for replying. So on easy note, i would have to assign those machines the preferred dns and use