Re: obj.ttl derived?
On Mon, Sep 14, 2009 at 12:52:15PM -0700, David Birdsong wrote: (snip snip) > > But are you sure you're getting hits at all? > > varnistat shows hitrate around .54 for 10 and 1000 second averages. N > expired objects is anywhere between 10-40 second though and varnishd > has only been running for about 12 hours. i expect the hit rate to > drop sharply as peak traffic continues to diminish for the day. Depending on the content, that might be a very low hit-rate. > i think a 9000s obj.ttl is a complete mistake on our part. so i've > followed some of the steps found here > http://varnish.projects.linpro.no/wiki/VCLExampleLongerCaching > on 1 of our servers. i'm going to compare it in a day or so. > > why do you ask though? It's fairly common to have pages that aren't cached. If I'm going to give you a single tip right now, it's to look at the output of: varnishtop -i TxURL That's a sorted list of what urls Varnish requests from your backend (in other words: cache misses). Most items should have a 1 or less next to them. The number represents a decaying average, and you should know why if you have items there that have double-digit representation. It should also give you an indication of whether this is a problem with your TTL or a few pages that aren't cached at all, all though the pass/hitpass counter in varnishstat will tell you that too. -- Kristian Lyngstøl Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 pgpAbFlAYDjWN.pgp Description: PGP signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: obj.ttl derived?
On Mon, Sep 14, 2009 at 12:45 PM, Kristian Lyngstol
wrote:
> On Mon, Sep 14, 2009 at 12:43:20PM -0700, David Birdsong wrote:
>> On Mon, Sep 14, 2009 at 12:38 PM, David Birdsong
>> wrote:
>> > awesome, thanks. this explains poor cache-hit ratio.
>> >
>> > On Mon, Sep 14, 2009 at 12:22 PM, Kristian Lyngstol
>> > wrote:
>> >> On Mon, Sep 14, 2009 at 11:57:25AM -0700, David Birdsong wrote:
>> >>> How is obj.ttl derived when both Expires and max-age is set by the
>> >>> backends?
>> >>>
>> >>> We had a case where the backend was setting Expires to 60seconds after
>> >>> the request and max-age was 5184000. Additionally in vcl_fetch:
>> >>>
>> >>> sub vcl_fetch {
>> >>> if (obj.ttl < 9000s) {
>> >>> set obj.ttl = 9000s;
>> >>> }
>> >>> }
>> >>>
>> >>> What would obj.ttl be set to given the Expires and max-age contradiction?
>> >>
>> >> If s-maxage is set, use that as default, otherwise use max-age as default.
>> >>
>> >> If Expires is sooner than the default, use that.
>> >>
>> >> So in your example, the ttl should be 60s.
>> oh, to clarify...
>>
>> would the stanza in vcl_fetch override the obj.ttl which was set by
>> the Expire header?
>> sub vcl_fetch {
>> if (obj.ttl < 9000s) {
>> set obj.ttl = 9000s;
>> }
>> }
>>
>> or do I have the order wrong?
>
> Ah, yes, it would, I misread the < as >, sorry about that.
>
> But are you sure you're getting hits at all?
varnistat shows hitrate around .54 for 10 and 1000 second averages. N
expired objects is anywhere between 10-40 second though and varnishd
has only been running for about 12 hours. i expect the hit rate to
drop sharply as peak traffic continues to diminish for the day.
i think a 9000s obj.ttl is a complete mistake on our part. so i've
followed some of the steps found here
http://varnish.projects.linpro.no/wiki/VCLExampleLongerCaching
on 1 of our servers. i'm going to compare it in a day or so.
why do you ask though?
>
> --
> Kristian Lyngstøl
> Redpill Linpro AS
> Tlf: +47 21544179
> Mob: +47 99014497
>
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: obj.ttl derived?
On Mon, Sep 14, 2009 at 12:43:20PM -0700, David Birdsong wrote:
> On Mon, Sep 14, 2009 at 12:38 PM, David Birdsong
> wrote:
> > awesome, thanks. this explains poor cache-hit ratio.
> >
> > On Mon, Sep 14, 2009 at 12:22 PM, Kristian Lyngstol
> > wrote:
> >> On Mon, Sep 14, 2009 at 11:57:25AM -0700, David Birdsong wrote:
> >>> How is obj.ttl derived when both Expires and max-age is set by the
> >>> backends?
> >>>
> >>> We had a case where the backend was setting Expires to 60seconds after
> >>> the request and max-age was 5184000. Additionally in vcl_fetch:
> >>>
> >>> sub vcl_fetch {
> >>> if (obj.ttl < 9000s) {
> >>> set obj.ttl = 9000s;
> >>> }
> >>> }
> >>>
> >>> What would obj.ttl be set to given the Expires and max-age contradiction?
> >>
> >> If s-maxage is set, use that as default, otherwise use max-age as default.
> >>
> >> If Expires is sooner than the default, use that.
> >>
> >> So in your example, the ttl should be 60s.
> oh, to clarify...
>
> would the stanza in vcl_fetch override the obj.ttl which was set by
> the Expire header?
> sub vcl_fetch {
>if (obj.ttl < 9000s) {
>set obj.ttl = 9000s;
>}
> }
>
> or do I have the order wrong?
Ah, yes, it would, I misread the < as >, sorry about that.
But are you sure you're getting hits at all?
--
Kristian Lyngstøl
Redpill Linpro AS
Tlf: +47 21544179
Mob: +47 99014497
pgpEcbDgVNBo8.pgp
Description: PGP signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: obj.ttl derived?
On Mon, Sep 14, 2009 at 12:38 PM, David Birdsong
wrote:
> awesome, thanks. this explains poor cache-hit ratio.
>
> On Mon, Sep 14, 2009 at 12:22 PM, Kristian Lyngstol
> wrote:
>> On Mon, Sep 14, 2009 at 11:57:25AM -0700, David Birdsong wrote:
>>> How is obj.ttl derived when both Expires and max-age is set by the backends?
>>>
>>> We had a case where the backend was setting Expires to 60seconds after
>>> the request and max-age was 5184000. Additionally in vcl_fetch:
>>>
>>> sub vcl_fetch {
>>> if (obj.ttl < 9000s) {
>>> set obj.ttl = 9000s;
>>> }
>>> }
>>>
>>> What would obj.ttl be set to given the Expires and max-age contradiction?
>>
>> If s-maxage is set, use that as default, otherwise use max-age as default.
>>
>> If Expires is sooner than the default, use that.
>>
>> So in your example, the ttl should be 60s.
oh, to clarify...
would the stanza in vcl_fetch override the obj.ttl which was set by
the Expire header?
sub vcl_fetch {
if (obj.ttl < 9000s) {
set obj.ttl = 9000s;
}
}
or do I have the order wrong?
>>
>> --
>> Kristian Lyngstøl
>> Redpill Linpro AS
>> Tlf: +47 21544179
>> Mob: +47 99014497
>>
>
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: obj.ttl derived?
awesome, thanks. this explains poor cache-hit ratio.
On Mon, Sep 14, 2009 at 12:22 PM, Kristian Lyngstol
wrote:
> On Mon, Sep 14, 2009 at 11:57:25AM -0700, David Birdsong wrote:
>> How is obj.ttl derived when both Expires and max-age is set by the backends?
>>
>> We had a case where the backend was setting Expires to 60seconds after
>> the request and max-age was 5184000. Additionally in vcl_fetch:
>>
>> sub vcl_fetch {
>> if (obj.ttl < 9000s) {
>> set obj.ttl = 9000s;
>> }
>> }
>>
>> What would obj.ttl be set to given the Expires and max-age contradiction?
>
> If s-maxage is set, use that as default, otherwise use max-age as default.
>
> If Expires is sooner than the default, use that.
>
> So in your example, the ttl should be 60s.
>
> --
> Kristian Lyngstøl
> Redpill Linpro AS
> Tlf: +47 21544179
> Mob: +47 99014497
>
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: obj.ttl derived?
On Mon, Sep 14, 2009 at 11:57:25AM -0700, David Birdsong wrote:
> How is obj.ttl derived when both Expires and max-age is set by the backends?
>
> We had a case where the backend was setting Expires to 60seconds after
> the request and max-age was 5184000. Additionally in vcl_fetch:
>
> sub vcl_fetch {
> if (obj.ttl < 9000s) {
> set obj.ttl = 9000s;
> }
> }
>
> What would obj.ttl be set to given the Expires and max-age contradiction?
If s-maxage is set, use that as default, otherwise use max-age as default.
If Expires is sooner than the default, use that.
So in your example, the ttl should be 60s.
--
Kristian Lyngstøl
Redpill Linpro AS
Tlf: +47 21544179
Mob: +47 99014497
pgpOkPH3GXLBG.pgp
Description: PGP signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
obj.ttl derived?
How is obj.ttl derived when both Expires and max-age is set by the backends?
We had a case where the backend was setting Expires to 60seconds after
the request and max-age was 5184000. Additionally in vcl_fetch:
sub vcl_fetch {
if (obj.ttl < 9000s) {
set obj.ttl = 9000s;
}
}
What would obj.ttl be set to given the Expires and max-age contradiction?
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: smart health check response?
In message , David Birdsong writes: >...are any of the stats available inside vcl_error? Not directly, but you can relatively easily get your hands on them if you use inline C-code. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish redundancy
On Mon, Sep 14, 2009 at 05:49:38PM +0100, Laurence Rowe wrote:
> 2009/9/3 Poul-Henning Kamp :
>
> > For it to be really smart you want to use directors for the
> > "other_varnish" and probes to ascertain health.
> >
> > We do not have a "priority_director" (we probably should have)
> > but you can get much the same effect with the random director
> > and very uneven weights:
> >
> > director other_backend {
> > { .backend = b_other_varnish ; weight=10; }
> > { .backend = b_real_backend ; weight=1; }
> > }
> >
> > Should the probes mark the other_varnish unhealthy, all trafic
> > will go to the real backend.
>
> Is there an advantage in using a director here instead of the following?
>
> sub vcl_recv {
> set req.backend = haproxy01;
> if (!req.backend.healthy) {
> set req.backend = haproxy02;
> }
Both approaches have their benefits.
One benefit of using a random director is that it can have multiple
fallbacks, but then again, you can achieve that by having two directors: a
pool of primary directors (or a single backend), and do
if (!req.backend.healthy) {
set req.backend = fallbackdirector;
}
Using a single director with the weight-approach does have a benefit of
being nicer to read and maintain, but will give the fallback backends some
marginal amount of traffic even when the primary backends are healthy.
--
Kristian Lyngstøl
Redpill Linpro AS
Tlf: +47 21544179
Mob: +47 99014497
pgpxKuIb1JxH9.pgp
Description: PGP signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish redundancy
2009/9/3 Poul-Henning Kamp :
> For it to be really smart you want to use directors for the
> "other_varnish" and probes to ascertain health.
>
> We do not have a "priority_director" (we probably should have)
> but you can get much the same effect with the random director
> and very uneven weights:
>
> director other_backend {
> { .backend = b_other_varnish ; weight=10; }
> { .backend = b_real_backend ; weight=1; }
> }
>
> Should the probes mark the other_varnish unhealthy, all trafic
> will go to the real backend.
Is there an advantage in using a director here instead of the following?
sub vcl_recv {
set req.backend = haproxy01;
if (!req.backend.healthy) {
set req.backend = haproxy02;
}
...
Laurence
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish Connection: close and IE6
Václav Bílek napsal(a): > Hello > > > We are trying to deploy varnish in production but IE6 is a big problem > for us. > > On the first try of lounching varnish we learned that it is imposible to > use keepalive because of the number of clients. So we tried to disable > keepalive by adding : > > set resp.http.Connection="close"; > > to the deliver. All worked fine except of IE6, where page load time > raise extremely. So we tried to disable keepalive:off for IE6 by piping > it through, but the pipe in varnish is probably buggy we get many > asserts and varnish became unusable. > Then web investigate further what is the reason of IE slow pageloads... > the reason is that IE on some objects (for example png), doesn't care > the "Connection: close" and doesn't close connection after getting > content, then after varnish session timeout varnish closes the > connection and IE gets another img and waits and so on... > > > Question: > > Is there any way around that? Helo With knowledge of that we dont know exactly how to patch for disabling keepalive we tried nasty hack: diff bin/varnishdcache_acceptor_epoll.c bin/varnishdcache_acceptor_epoll.c.new 114c114 < deadline = TIM_real() - params->sess_timeout; --- > //deadline = TIM_real() - params->sess_timeout; 117c117 < if (sp->t_open > deadline) --- > //if (sp->t_open > deadline) it worked in testing enviroment but in real trafic it was even worse (IE6 hanging for long time). I will be glad for any advice. Thanks Vaclav Bilek ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
