On 02/09/2017 04:23 PM, William Tu wrote:
On Thu, Feb 9, 2017 at 5:11 AM, Daniel Borkmann <[email protected]> wrote:
On 02/08/2017 09:22 PM, William Tu via iovisor-dev wrote:
Hi,
I have a program which I use around at most 300byte of local stack as
below. The largest struct is the "struct Headers" which is around 80
byte. However, when loading into the verifier, it says
393: (7b) *(u64 *)(r10 -56) = r1
394: (05) goto pc+56
451: (7b) *(u64 *)(r10 -528) = r0
invalid stack off=-528 size=8
I don't think I'm using more than 512byte. It seems that the llvm
generates the code which use more stack memory than I thought. Any
idea how to debug it? Or how to dump the llvm IR to know how it
allocates stack? Thanks
snippet of the code:
SEC("prog")
int ebpf_filter(struct xdp_md* skb){
struct Headers hd;
unsigned ebpf_packetOffsetInBits = 0;
enum ebpf_errorCodes ebpf_errorCode = NoError;
void* ebpf_packetStart = ((void*)(long)skb->data);
void* ebpf_packetEnd = ((void*)(long)skb->data_end);
u32 ebpf_zero = 0;
u8 ebpf_byte = 0;
u32 ebpf_outHeaderLength = 0;
struct xdp_output xout;
/* TODO: this should be initialized by the environment. HOW? */
struct xdp_input xin;
goto start;
start: {
/* extract(hd.ethernet
Looks like a code generation issue perhaps?
yes, I guess LLVM doesn't like the pattern we generate, I will try
different patterns to see if it makes any difference.
You are generating the C files in the end, right? So the code
shown is what clang gets to compile; or does this come from
another llvm front end somehow?
Also this below looks pretty odd, I'd have expected LLVM to
either just xor r5 with itself or set it to 0 _once_, and not
every time it clears some stack mem (oob aside).
[...]
78: r5 = 0
79: *(u64 *)(r10 - 256) = r5
80: r5 = 0
81: *(u64 *)(r10 - 264) = r5
82: r5 = 0
83: *(u64 *)(r10 - 240) = r5
84: r5 = 0
85: *(u64 *)(r10 - 248) = r5
86: r5 = 0
87: *(u64 *)(r10 - 224) = r5
88: r5 = 0
89: *(u64 *)(r10 - 232) = r5
[...]
It seems your prog parses and fills a huge struct on the stack, f.e. eth dest
is filled from
direct packet acces byte by byte (inefficient, but fair enough). The
The performance of byte-by-byte load will be bad, but we decide to
work on correctness first.
below annotation somehow seems to be off slightly (?), but it's always
patterns like:
ya. it is off by 1 line.
I do try packing the structure, using
#pragma pack(1)
but makes no difference.
Thanks for your reply~
--William
r1 = *(u8 *)(r2 + 11) ; load byte 1 from pkt into reg (u8)
*(u64 *)(r10 - 408) = r1 ; store byte 1 into stack (u64)
r1 = *(u8 *)(r2 + 10) ; load byte 2 from pkt into reg (u8)
*(u64 *)(r10 - 416) = r1 ; store byte 2 into stack (u64)
[...]
; void* ebpf_packetStart = ((void*)(long)skb->data);
2: r2 = *(u32 *)(r6 + 0)
[...]
; hd.ethernet.destination[5] = (u8)((load_byte(ebpf_packetStart,
BYTES(ebpf_packetOffsetInBits) + 5) >> 0));
9: r1 = *(u8 *)(r2 + 11)
; hd.ethernet.destination[4] = (u8)((load_byte(ebpf_packetStart,
BYTES(ebpf_packetOffsetInBits) + 4) >> 0));
10: *(u64 *)(r10 - 408) = r1
11: r1 = *(u8 *)(r2 + 10)
; hd.ethernet.destination[3] = (u8)((load_byte(ebpf_packetStart,
BYTES(ebpf_packetOffsetInBits) + 3) >> 0));
12: *(u64 *)(r10 - 416) = r1
13: r1 = *(u8 *)(r2 + 9)
; hd.ethernet.destination[2] = (u8)((load_byte(ebpf_packetStart,
BYTES(ebpf_packetOffsetInBits) + 2) >> 0));
14: *(u64 *)(r10 - 424) = r1
15: r1 = *(u8 *)(r2 + 8)
; hd.ethernet.destination[1] = (u8)((load_byte(ebpf_packetStart,
BYTES(ebpf_packetOffsetInBits) + 1) >> 0));
16: *(u64 *)(r10 - 432) = r1
17: r1 = *(u8 *)(r2 + 7)
; hd.ethernet.destination[0] = (u8)((load_byte(ebpf_packetStart,
BYTES(ebpf_packetOffsetInBits) + 0) >> 0));
18: *(u64 *)(r10 - 440) = r1
19: r1 = *(u8 *)(r2 + 6)
[...]
Despite a struct of:
struct Ethernet {
char source[6]; /* bit<48> */
char destination[6]; /* bit<48> */
u16 protocol; /* bit<16> */
u8 ebpf_valid;
};
Maybe packing structs helps a bit (but still shouldn't be such waste,
hmm ...).
full C, objdump
https://gist.github.com/williamtu/5a09b60a951ee5fc062328766403ab4b
thanks
_______________________________________________
iovisor-dev mailing list
[email protected]
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
_______________________________________________
iovisor-dev mailing list
[email protected]
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
