Re: bird: memory exhausted

[Job Snijders]

On Mon, Aug 07, 2017 at 03:01:49AM +0200, Ondrej Zajicek wrote:
> > > Any ideas how to increase the memory available to things on the stack?
> > 
> > Looks like you're hitting the Bison parser stack limit. The default
> > is 1, which is probably too low, looking at your sample config
> > file.
> 
> It is true that there is a stack limit, but the core problem is that
> involved grammar rules use right recursion instead of left recursion
> like most other rules. That causes excessive stack usage.
> 
> Workaround could to use global definitions instead of local variables, i.e:
> 
> define AS1 = [ 170.238.64.0/23, 152.231.29.0/24 ];
> 
> Also note that instead of sequence of ifs it is much more efficient to
> use case expression based on asn and then just net-based test inside of
> case branch:
> 
> case asn {
>   0: return net ~ AS0;
>   1: return net ~ AS1;
>   2: return net ~ AS2;
>   ...
> }

I implemented your approach, this indeed seesm to be an optimalisation.

Thank you! :)

Kind regards,

Job

Re: bird: memory exhausted

[Ondrej Zajicek]

On Sun, Aug 06, 2017 at 10:52:27PM +0200, Andre van Zyl wrote:
> On Sun, 6 Aug 2017 16:24:24 +0200
> Job Snijders  wrote:
> 
> > Dear all,
> > 
> > It appears I'm hitting some kind of limit in yacc stuff (on 
> > 1.6.3-1+trusty+1):
> > 
> > router# bird -c rpki-match-roa-ipv4.conf  -p
> > bird: rpki-match-roa-ipv4.conf, line 4998: memory exhausted
> > 
> > The error message is generated by generated code:
> > 
> > $ grep -r 'memory exhausted' *
> > Binary file bird matches
> > obj/conf/cf-parse.tab.c:yyoverflow (YY_("memory exhausted"),
> > obj/conf/cf-parse.tab.c:  yyerror (YY_("memory exhausted"));
> > Binary file obj/conf/all.o matches
> > Binary file obj/conf/cf-parse.tab.o matches
> > 
> > The config file can be found here: 
> > http://instituut.net/~job/rpki-match-roa-ipv4.conf.txt
> > 
> > Any ideas how to increase the memory available to things on the stack?
> > 
> > Kind regards,
> > 
> > Job
> 
> Hi Job,
> 
> Looks like you're hitting the Bison parser stack limit. The default is 1, 
> which is probably too low, looking at your sample config file.

Hi

It is true that there is a stack limit, but the core problem is that
involved grammar rules use right recursion instead of left recursion like
most other rules. That causes excessive stack usage.

Workaround could to use global definitions instead of local variables, i.e:

define AS1 = [ 170.238.64.0/23, 152.231.29.0/24 ];

Also note that instead of sequence of ifs it is much more efficient to
use case expression based on asn and then just net-based test inside of
case branch:

case asn {
  0: return net ~ AS0;
  1: return net ~ AS1;
  2: return net ~ AS2;
  ...
}

-- 
Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santi...@crfreenet.org)
OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
"To err is human -- to blame it on a computer is even more so."


signature.asc
Description: PGP signature

Re: bird: memory exhausted

[Andre van Zyl]

On Sun, 6 Aug 2017 16:24:24 +0200
Job Snijders  wrote:

> Dear all,
> 
> It appears I'm hitting some kind of limit in yacc stuff (on 1.6.3-1+trusty+1):
> 
>   router# bird -c rpki-match-roa-ipv4.conf  -p
>   bird: rpki-match-roa-ipv4.conf, line 4998: memory exhausted
> 
> The error message is generated by generated code:
> 
>   $ grep -r 'memory exhausted' *
>   Binary file bird matches
>   obj/conf/cf-parse.tab.c:yyoverflow (YY_("memory exhausted"),
>   obj/conf/cf-parse.tab.c:  yyerror (YY_("memory exhausted"));
>   Binary file obj/conf/all.o matches
>   Binary file obj/conf/cf-parse.tab.o matches
> 
> The config file can be found here: 
> http://instituut.net/~job/rpki-match-roa-ipv4.conf.txt
> 
> Any ideas how to increase the memory available to things on the stack?
> 
> Kind regards,
> 
> Job

Hi Job,

Looks like you're hitting the Bison parser stack limit. The default is 1, 
which is probably too low, looking at your sample config file.

You can increase the stack limit by defining YYMAXDEPTH in obj/conf/config.Y 
like the diff below. Raising this arbitrarily to 10 seem to fix the error 
for me, but I have no idea what a sane value is here, so you might get a better 
value through trial and error.

Regards,
Andre

--- obj/conf/config.Y   2017-08-06 22:34:05.309017152 +0200
+++ ../config.Y 2017-08-06 22:35:05.041017152 +0200
@@ -11,6 +11,8 @@
 #include "lib/unix.h"
 #include 
 
+#define YYMAXDEPTH 10
+
 CF_DECLS
 
 CF_KEYWORDS(LOG, SYSLOG, ALL, DEBUG, TRACE, INFO, REMOTE, WARNING, ERROR, 
AUTH, FATAL, BUG, STDERR, SOFT)