Re: GRSEC and Varnish

[Bernardf FRIT]

Kristian Lyngstol a écrit :
 On Tue, Feb 02, 2010 at 04:44:48PM +0100, Bernardf FRIT wrote:
 Hi,

 I'am running :
 - varnishd (varnish-2.0.4)

 Why not 2.0.6?
When a server is running well, I'm a bit reluctant to upgrade. Now, I'm
ok to upgrade as an attempt to fix this.

 and it appears that the grsec Kernel repeatedly and unexpectedly sends 
 signal 11 to the varnishd child.

 grsec seems to just report that a segfault occurred. SIGSEG would be a
 strange signal to send in any event. You want to fetch yourself a core-dump
 of this. However, before we get into that, I'd like to know what parameters
 you start Varnish with, and the general setup. VCL too, if possible.

Ok, I just misunderstood the grsec report. I can't find any core dump
file in the system.

I start varnishd using /etc/init.d/varnishd with the following parameters :

# cat /etc/conf.d/varnishd
# /etc/conf.d/varnishd

# options passed to varnish on startup
# please see the varnishd man page for more options
VARNISHD_OPTS=-a 87.98.137.117:80 -f /etc/varnish/yourimmo.vcl -n
/home/varnish/yourimmo -s file,/home/varnish/cache/yourimmo,1G -T
127.0.0.1:

# arguments passed to varnishncsa
# please see the varnishncsa man page for more options
VARNISHNCSA_ARGS=-c -a -n /home/varnish/yourimmo -w
/var/log/varnish/access.log

---

# cat /etc/varnish/yourimmo.vcl
### define backends:

# ha proxy
backend ha_proxy {
.host = 127.0.0.1;
.port = 80;
}

acl purge {
localhost;
111.111.111.111;
}

### Called when a client request is received

sub vcl_recv {

### if there is a purge make sure its coming from $localhost

if (req.request == PURGE) {
if (!client.ip ~ purge) {
error 405 Not allowed.;
}
lookup;
}

# Add a unique header containing the client address
remove req.http.X-Forwarded-For;
setreq.http.X-Forwarded-For = client.ip;

# setreq.http.X-Forwarded-For = req.http.rlnclientipaddr;

# grace settings, note this is also set in vcl_fetch,
set req.grace = 600s;

if (req.http.host ~ ^(www.)?your-immo.fr$) {
set req.backend = ha_proxy;
}

### ne pas mettre en cache:

if (req.request == GET  req.url ~ \.(php|html)$) {
pass;
}
if (req.request == GET  req.url ~ \.(your-immo\.fr)$) {
pass;
}

### toujours mettre en cache:

 if (req.request == GET  req.url ~ \.(js)) {
lookup;
}

## images
if (req.request == GET  req.url ~
\.(gif|jpg|jpeg|bmp|png|tiff|tif|ico|img|tga|wmf)$) {
lookup;
}

## pages statiques
if (req.request == GET  req.url ~ \.(css|pdf|exe)$) {
lookup;
}

  ## multimedia
if (req.request == GET  req.url ~
\.(svg|swf|ico|mp3|mp4|m4a|ogg|mov|avi|wmv)$) {
lookup;
}

## xml

if (req.request == GET  req.url ~ \.(xml)$) {
lookup;
}

### regles a  ne pas mettre en cache:

if (req.request == GET  req.url ~ \/stats) {
 pipe;
}
if (req.request != GET  req.request != HEAD) {
pipe;
}
if (req.http.Authenticate || req.http.Authorization) {
pass;
}

###  ne pas mettre en cache les sessions d'authenticfication
if (req.http.Cookie  req.http.Cookie ~ authtoken=) {
pipe;
}


### parse accept encoding rulesets to make it look nice
if (req.http.Accept-Encoding) {
if (req.http.Accept-Encoding ~ gzip) {
set req.http.Accept-Encoding = gzip;
} elsif (req.http.Accept-Encoding ~ deflate) {
set req.http.Accept-Encoding = deflate;
} else {
# unkown algorithm
remove req.http.Accept-Encoding;
}
}

### Modif suite a segfault
pass;

### if it passes all these tests, do a lookup anyway;
###lookup;

### end of vcl_recv
}

### Called when an object is in the cache, its a hit.
sub vcl_hit {
  if (req.request == PURGE) {
  set obj.ttl = 0s;
  error 200 Purged.;
  }
  if (!obj.cacheable) {
  pass;
  }

deliver;
}

### Called when the requested object was not found in the cache

sub vcl_miss {
  if (req.request == PURGE) {
  error 404 Not in cache.;
  }
}

### Called when the requested object has been retrieved from the
backend, or the request to the backend has failed

sub vcl_fetch {

## If the request to the backend returns a code other than 200, restart
the loop
## If the number of restarts reaches the value of the parameter
max_restarts,
## the request will be error'ed.  max_restarts defaults to 4.  This
prevents
## an eternal loop in the event that, e.g., the object does not exist
at all.
## this rule also allows for 301's and 302's redirects...

Re: GRSEC and Varnish

[Kristian Lyngstol]

On Tue, Feb 02, 2010 at 04:44:48PM +0100, Bernardf FRIT wrote:
 Hi,
 
 I'am running :
 - varnishd (varnish-2.0.4)

Why not 2.0.6?

 and it appears that the grsec Kernel repeatedly and unexpectedly sends 
 signal 11 to the varnishd child.

grsec seems to just report that a segfault occurred. SIGSEG would be a
strange signal to send in any event. You want to fetch yourself a core-dump
of this. However, before we get into that, I'd like to know what parameters
you start Varnish with, and the general setup. VCL too, if possible.

-- 
Kristian Lyngstøl
Redpill Linpro AS
Mob: +47 99014497


pgpFwMaQMo7wh.pgp
Description: PGP signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc

Re: GRSEC and Varnish

[Mark Moseley]

On Thu, Feb 4, 2010 at 7:59 AM, Bernardf FRIT bern...@frit.net wrote:
 Mark Moseley a écrit :

 grsec will often report that signals were sent, not that grsec
 necessarily sent that signal itself. I don't think I've ever actually
 seen it report itself sending a signal to a process. So varnishd could
 be segfaulting for some reason and grsec is just reporting this. If
 you're getting a core file, try loading it into gdb and using 'bt' on
 it, to see where it's dying.

 I cannot get any core file.

 One other thing to try: As soon as it happens, try using 'dmesg' (or
 dmesg -s 131072 in case you have lots of things logging to the
 kernel logs) and grep for PAX. It's not likely, but PAX could be
 killing it due to some violation. And for whatever reason, the PAX
 message doesn't show up in the logs, just in dmesg.

 Nothing more in dmesg than in kern.log. ;-[

 varnishd[27069]: segfault at 1000 ip 0043abf0 sp 45696ae0
 error 4 in varnishd[40+5]
 grsec: From 80.13.9.224: signal 11 sent to
 /usr/sbin/varnishd[varnishd:27069] uid/euid:65534/65534
 gid/egid:65534/65534, parent /usr/sbin/varnishd[varnishd:11327] uid/euid:0/0
 gid/egid:0/0
 varnishd[27992]: segfault at 1000 ip 0043abf0 sp 48140ae0
 error 4 in varnishd[40+5]
 grsec: From 90.4.162.78: signal 11 sent to
 /usr/sbin/varnishd[varnishd:27992] uid/euid:65534/65534
 gid/egid:65534/65534, parent /usr/sbin/varnishd[varnishd:11327] uid/euid:0/0
 gid/egid:0/0
 varnishd[28119]: segfault at ed000 ip 0043abf0 sp 47ceaae0
 error 4 in varnishd[40+5]
 grsec: From 90.44.219.18: signal 11 sent to
 /usr/sbin/varnishd[varnishd:28119] uid/euid:65534/65534
 gid/egid:65534/65534, parent /usr/sbin/varnishd[varnishd:11327] uid/euid:0/0
 gid/egid:0/0

Try putting ulimit -c unlimited in your varnishd rc file. I haven't
needed to get a varnishd core file before, so maybe the devs might be
able to advise if there's other steps necessary as well. There should
also be some logs saying that it died (or at least that it restarted);
dunno what your distro you're using, but in debian, those typically
end up in /var/log/syslog.

You could tail varnishncsa to see if there's a common request where it
seems to segfault at and if there is, you could attach to varnishd
with gdb /path/to/varnishd pid of varnishd and try to trigger it.
Then get the backtrace with 'bt'. But be aware that it'll bog it down
dreadfully, so i wouldn't advise it in production.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc

Re: GRSEC and Varnish

[Mark Moseley]

On Tue, Feb 2, 2010 at 11:53 PM, Tollef Fog Heen
tfh...@varnish-software.com wrote:
 ]] Bernardf FRIT

 | Then the parent varnishd process starts immediately a new child process
 | which lasts some time.
 |
 | Is there any way to fix this. Remocve the GRSEC kernel ? Upgrade the
 | kernel ? Varnish ? or whatever ?

 Work out why it thinks that varnishd is doing something wrong?  It
 doesn't seem to say so in the log.

 --
 Tollef Fog Heen
 Redpill Linpro -- Changing the game!
 t: +47 21 54 41 73
 ___
 varnish-misc mailing list
 varnish-misc@projects.linpro.no
 http://projects.linpro.no/mailman/listinfo/varnish-misc


grsec will often report that signals were sent, not that grsec
necessarily sent that signal itself. I don't think I've ever actually
seen it report itself sending a signal to a process. So varnishd could
be segfaulting for some reason and grsec is just reporting this. If
you're getting a core file, try loading it into gdb and using 'bt' on
it, to see where it's dying.

One other thing to try: As soon as it happens, try using 'dmesg' (or
dmesg -s 131072 in case you have lots of things logging to the
kernel logs) and grep for PAX. It's not likely, but PAX could be
killing it due to some violation. And for whatever reason, the PAX
message doesn't show up in the logs, just in dmesg.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc

GRSEC and Varnish

[Bernardf FRIT]

Hi,

I'am running :
- varnishd (varnish-2.0.4)
- linux kernel 2.6.27.10-grsec--grs-ipv4-64

and it appears that the grsec Kernel repeatedly and unexpectedly sends 
signal 11 to the varnishd child.

.../...
Feb  2 12:01:02 XX varnishd[17111]: segfault at 1000 ip 
0043abf0 sp 47d89ae0 error 4 in varnishd[40+5]
Feb  2 12:01:02 XX grsec: From 82.67.39.69: signal 11 sent to 
/usr/sbin/varnishd[varnishd:17111] uid/euid:65534/65534 
gid/egid:65534/65534, parent
 /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0
Feb  2 13:45:44 XX varnishd[22187]: segfault at f5000 ip 
0043abf0 sp 48538ae0 error 4 in varnishd[40+5]
Feb  2 13:45:44 XX grsec: From 80.13.19.228: signal 11 sent to 
/usr/sbin/varnishd[varnishd:22187] uid/euid:65534/65534 
gid/egid:65534/65534, paren
t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0
Feb  2 13:54:57 XX varnishd[22236]: segfault at 1000 ip 
0043abf0 sp 45445ae0 error 4 in varnishd[40+5]
Feb  2 13:54:57 XX grsec: From 80.13.19.228: signal 11 sent to 
/usr/sbin/varnishd[varnishd:22236] uid/euid:65534/65534 
gid/egid:65534/65534, paren
t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0
Feb  2 14:13:41 XX varnishd[22595]: segfault at ae000 ip 
0043abf0 sp 40ff4ae0 error 4 in varnishd[40+5]
Feb  2 14:13:41 XX grsec: From 83.145.80.130: signal 11 sent to 
/usr/sbin/varnishd[varnishd:22595] uid/euid:65534/65534 
gid/egid:65534/65534, pare
nt /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0
Feb  2 14:31:08 XX varnishd[23547]: segfault at 1000 ip 
0043abf0 sp 45b40ae0 error 4 in varnishd[40+5]
Feb  2 14:31:08 XX grsec: From 81.49.118.48: signal 11 sent to 
/usr/sbin/varnishd[varnishd:23547] uid/euid:65534/65534 
gid/egid:65534/65534, paren
t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0
Feb  2 16:19:05 XX varnishd[24256]: segfault at f7000 ip 
0043abf0 sp 473bcae0 error 4 in varnishd[40+5]
Feb  2 16:19:05 XX grsec: From 192.196.142.20: signal 11 sent to 
/usr/sbin/varnishd[varnishd:24256] uid/euid:65534/65534 
gid/egid:65534/65534, par
ent /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0

Then the parent varnishd process starts immediately a new child process 
which lasts some time.

Is there any way to fix this. Remocve the GRSEC kernel ? Upgrade the 
kernel ? Varnish ? or whatever ?

Thanks in advance.
--
Bernard FRIT



___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc

Re: GRSEC and Varnish

[Tollef Fog Heen]

]] Bernardf FRIT 

| Then the parent varnishd process starts immediately a new child process 
| which lasts some time.
| 
| Is there any way to fix this. Remocve the GRSEC kernel ? Upgrade the 
| kernel ? Varnish ? or whatever ?

Work out why it thinks that varnishd is doing something wrong?  It
doesn't seem to say so in the log.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc