Re: Varnish logging and data merging
Thanks again Rob! Tracking pixel it is then :) Off to cobble some code together and test. On Mon, Jan 11, 2010 at 3:42 AM, Rob S wrote: > pub crawler wrote: >> >> We have a lot of logging going on in our applications. Logs pages, IP >> info, time date, URL parameters, etc. Since many pages are being >> served out of Varnish cache, they don't get logged by our >> application. >> >> How is anyone else out there working around this sort of problem with >> an existing application? Considering a 1x1 graphic file inclusion >> into our pages to facilitate logging and ensuring Varnish doesn't >> cache it > > We've not reported analytics based on raw Apache logs for a long time - > they're far too polluted by spiders etc. So, instead, for RSS we use > Feedburner (who provide stats, and reduce our traffic load), and for general > web access we either use Google Analytics (or commercial alternatives). > Google Analytics allows you to add and report on custom parameters, so it > is very flexible. However, as you suggested, an alternative is to use a > tracking pixel. Depending on how you've previously processed your > information, you may find it very useful to encourage Varnish to route all > hits to your tracking pixel to a specific server (obviously with failover). > This'd save you having to aggregate logs across multiple servers. > > Rob > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish logging and data merging
pub crawler wrote: > We have a lot of logging going on in our applications. Logs pages, IP > info, time date, URL parameters, etc. Since many pages are being > served out of Varnish cache, they don't get logged by our > application. > > How is anyone else out there working around this sort of problem with > an existing application? Considering a 1x1 graphic file inclusion > into our pages to facilitate logging and ensuring Varnish doesn't > cache it We've not reported analytics based on raw Apache logs for a long time - they're far too polluted by spiders etc. So, instead, for RSS we use Feedburner (who provide stats, and reduce our traffic load), and for general web access we either use Google Analytics (or commercial alternatives). Google Analytics allows you to add and report on custom parameters, so it is very flexible. However, as you suggested, an alternative is to use a tracking pixel. Depending on how you've previously processed your information, you may find it very useful to encourage Varnish to route all hits to your tracking pixel to a specific server (obviously with failover). This'd save you having to aggregate logs across multiple servers. Rob ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish logging and data merging
In message <4c3149fb1001102237xaf0d732hb22418df4cf21...@mail.gmail.com>, pub cr awler writes: >We have a lot of logging going on in our applications. Logs pages, IP >info, time date, URL parameters, etc. Since many pages are being >served out of Varnish cache, they don't get logged by our >application. run varnishncsa or varnishlog to collect your loginfo. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish logging and data merging
Varnish does keep a log if you ask it to. On Jan 10, 2010, at 10:37 PM, pub crawler wrote: > Alright, up and running with Varnish successfully. Quite happy with > Varnish. Our app servers no longer are failing / overwhelmed. > > Here's our new problem... > > We have a lot of logging going on in our applications. Logs pages, IP > info, time date, URL parameters, etc. Since many pages are being > served out of Varnish cache, they don't get logged by our > application. > > How is anyone else out there working around this sort of problem with > an existing application? Considering a 1x1 graphic file inclusion > into our pages to facilitate logging and ensuring Varnish doesn't > cache it. > > Share your ideas. > > -Paul > ___ > varnish-misc mailing list > varnish-misc@projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish logging and data merging
Alright, up and running with Varnish successfully. Quite happy with Varnish. Our app servers no longer are failing / overwhelmed. Here's our new problem... We have a lot of logging going on in our applications. Logs pages, IP info, time date, URL parameters, etc. Since many pages are being served out of Varnish cache, they don't get logged by our application. How is anyone else out there working around this sort of problem with an existing application? Considering a 1x1 graphic file inclusion into our pages to facilitate logging and ensuring Varnish doesn't cache it. Share your ideas. -Paul ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
