On Fri, Apr 14, 2017 at 12:48:11PM -0700, William Tu wrote: > Thanks for all the feedbacks from the iovisor summit. Since then, > we've tried a couple of workarounds for this issue, the following > shares the observations and code. > > === Original code, xdp9.c === > xdp9.c is to parse IPv4, TCP, UDP, ICMP and update checksum. > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-xdp9-c > which generated from xdp9.p4 > https://github.com/vmware/p4c-xdp/blob/master/tests/xdp9.p4 > > its objdump, with max stack size = 600 > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-xdp9-objdump-L551 > > [Result]: fail to load into kernel > The culprit is "struct Headers hd" which is < 200 byte, but due to > register spill, it uses lots of stack. > > === Work around 1 === > Reduce the live range of the registers by declaring another "struct > Headers hd__", and use "hd__" at the lower half of the program. > > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-workaround1-xdp9-c > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-workaround1-xdp9-objdump > > [Result]: max stack size reduces to 384, runs OK, but this introduce extra > copy. > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-workaround1-xdp9-objdump-L404 > > === Work around 2 === > Save the "struct Headers hd" in a per cpu array map. > > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-workaround2-xdp9-c > > [Result]: since it's not using stack, max stack size < 200, program > runs OK. Not sure what's the performance overhead until we stress test > it. > > === Work around 3 === > (experimental) declare 'struct Headers hd' as 'volatile' > In this case, access to the 'hd' will be read/write from memory, > ideally should use less registers. > > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-volatile-xdp9-c > https://gist.github.com/williamtu/759eedbefa335c7dbc5700acc0c43ee9#file-volatile-xdp9-objdump > > [Result:] the generated code is always in order, the verifier hit the > max insn it can processed: > BPF program is too large. Processed 65537 insn > Error fetching program/map!
could you increase the max to 1M or so and see what the actual number of insns it needs? If the program is short, as you say, can you debug the state prunning to see why it's not recognized? I'm guessing stack slots are too different. If so, can you apply the same hueristics we have for registers to stack slots ? In other words apply upper half of states_equal() to lower half? _______________________________________________ iovisor-dev mailing list [email protected] https://lists.iovisor.org/mailman/listinfo/iovisor-dev
