Hi Dridi / Team, Gentle reminder..
Thank you Hardik On Thu, 4 Apr 2019 at 11:33, Hardik <hetardi...@gmail.com> wrote: > Thanks Dridi. > > I was ill and off work for few days so did not reply. > > Hey Dridi, I am already using varnishncsa. And use case perspective I am > much familiar. I am using below configuration for varnishncsa. This logs I > am calling as raw-logs/Apache and providing to customer directly without > any modification.Means no processing on this. > > #the format of the log lines > FORMAT='-F "%h %v %u %t \"%m %U%q %H\" %s %O \"%{Referer}i\" > \"%{User-agent}i\" %S [%{VCL_Log:x-product}x lg=\"%{Accept-Language}i\" > fwd=\"%{X-Forwarded-For}i\" %{VCL_Log:namespace}x > %{VCL_Log:x-cache-hit}x/%{VCL_Log:x-revalidate-cache}x > %{VCL_Log:sc-substatus}x] > > Note- few parameter I have added in varnishncsa to get that information > ex:%S. > > I have written other utility because want to parse all the fields based on > billing requirements. I have tried removing "-g session" option and now log > loss reduced a lot(I can say 95%). > > If you can help on my previous reply/doubts then will be really helpful ( > because billing really difficult to change billing setup).I am only facing > two issues here, 1. log loss 2. missing of ReqAcct ( which is also case in > varnishncsa). > > If I am doing something wrong other then "-g session" option in the code I > have pasted before from utility please point out. > > Thank you > Hardik > > On Wed, 20 Mar 2019 at 21:02, Dridi Boukelmoune <dr...@varni.sh> wrote: > >> On Wed, Mar 20, 2019 at 10:40 AM Hardik <hetardi...@gmail.com> wrote: >> > >> > Hi Dridi, >> > >> > Do you need all timestamps or a specific metric? >> > Regarding timestamp, want to read two tags, >> > Timestamp Start: 1516269224.184112 0.000000 0.000000 >> > Timestamp Resp: 1516269224.184920 0.000808 0.000087 >> > >> > Do you need the BereqAcct records for all transactions? Including cache >> hits? >> > Sorry it is my mistake. I am not reading any of the beck-end records. >> So can ignore BereqAcct. >> > I need fields from Req records only. >> >> Ok, in this case you can probably get away with just varnishncsa to >> collect all you need. >> >> No grouping (the default -g vxid), client mode (-c) only, with a >> custom -F format to grab only what you need. >> >> This should help reduce the churn to the point where you lose data. >> >> If you struggle with this, I can help you later with that, but start >> by reading the following manuals: >> >> - varnishncsa >> - vsl >> - vsl-query >> >> For example, the format for the timestamps you wish to collect would >> look like this: >> >> > %{VSL:Timestamp:Start[1]}x %{VSL:Timestamp:Resp[2]}x >> %{VSL:Timestamp:Resp[3]}x >> >> Rinse and repeat for all the things you need to capture for the logs, >> put them in the order you prefer and off you go. No need to write your >> own utility. >> >> > What does FD mean here? File descriptor? From ReqStart? >> > Yes, Its file descriptor. And yes reading from ReqStart till ReqAcct. >> Using switch case to read needed records. >> >> If you already work with VXIDs, the FD becomes redundant. >> >> >> >> Dridi >> >
_______________________________________________ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc