Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On Mon, Jun 22, 2015 at 03:55:01PM +0200, Daniel Borkmann wrote: On 06/21/2015 07:02 PM, Vadim Kochan wrote: ... Not sure if it is necessary to use libnet by trafgen as it is used by mz to generate packets ? It will add the new dependency, but seems it can simplify generation of the proto packets. Yes sure, we shouldn't depend on it, I think some small helper functions inside the netsniff-ng tree could do the job, too. Hm, I missed this email ... anyway I almost developed generic mechanism for proto fields generation which uses the low level trafgen's xxx_byte functions from trafgen_parser.c which I had to export, I have also implemented ethernet arp protocol generation based on the common field generation code, but I can't decide which command line syntax would be good, so I think that something like this might be ok (we should not follow mz style right?): $ trafgen --dev lo -n 1 eth sa=11:22:33:44:55:66 da=11:22:33:44:55:66, arp op=req tip=192.168.1.1 this syntax will allow to easy parse parameters per protocol name just by split whole cmdline by ,. Sure I tried to use mz's style but seems it more complicated: $ trafgen/trafgen --dev lo eth da = AA:BB:CC:DD:EE:FF,sa=11:22:33:44:55:66 arp tip=192.168.1.1 -n 1 by more complicated I mean that it will not allow to use strtok parsing but for each character, but sure it possible. mz seems parses params from the whole command line, but I'd like to separate params per protocol. Regards, Vadim Kochan -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On Sun, Jun 21, 2015 at 08:02:40PM +0300, Vadim Kochan wrote: On Sat, Jun 20, 2015 at 01:10:45AM +0200, Daniel Borkmann wrote: On 06/20/2015 12:25 AM, Vadim Kochan wrote: ... Ok, how would all of this tie together with flex and bison that we use for trafgen as parser generator? It looks on a first glance that this design would be outside of that scope? . You mean that this is out of scope of this thread which was related to mz ? If so, then yes ... may be extend the trafgen's cfg script to describe the protocols and then add the cmd line interface based on this extension ? Or do you think that extending cfg script syntax is not good or may be problematic ? Well, I think may be I can do some just for fun implementation, just for demonstration ... As a start, if you're interested, I think it would be great if you have some cycles to look into: http://thread.gmane.org/gmane.linux.network.netsniff-ng/799 I.e. the idea would be to have a _textual_ replacement initially and then push the config through the lexer/parser. Anyway, back to the topic, as it started out with mz. ;) I do quite like the interactive mode of mz and it's cmdline syntax is fine with me too. mz's implementation rather hacky and uneasy to maintain. trafgen so far has a very low level language on how to define packets. I'm happy to entertain the idea to add an mz-like _interactive_ mode that is libreadline-based with more or less similar feature set. I'm also good if we add a similar direct _cmdline_ usage possibility. Think of it as kind a /clean/ merge of mz which is in staging, into trafgen, together with the (f.e. performance) benefits of having direct packet socket access. Thus, we'd end up having the ability of cmdline, config, and interactive mode in one traffic generator with different levels of granularity. Thanks, Daniel Not sure if it is necessary to use libnet by trafgen as it is used by mz to generate packets ? It will add the new dependency, but seems it can simplify generation of the proto packets. Regards, Well, anyway I am trying to implement basic cmdline mz-like proto generation API with few layered protocols: eth, ip, udp for trafgen utility. If I success with this I will share the patch series (I will try to better split it for easier review). Regards, -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On 06/21/2015 07:02 PM, Vadim Kochan wrote: ... Not sure if it is necessary to use libnet by trafgen as it is used by mz to generate packets ? It will add the new dependency, but seems it can simplify generation of the proto packets. Yes sure, we shouldn't depend on it, I think some small helper functions inside the netsniff-ng tree could do the job, too. -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On 06/22/2015 03:37 PM, Vadim Kochan wrote: ... Well, anyway I am trying to implement basic cmdline mz-like proto generation API with few layered protocols: eth, ip, udp for trafgen utility. If I success with this I will share the patch series (I will try to better split it for easier review). Looking forward to review! -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On 06/19/2015 10:46 AM, Vadim Kochan wrote:
...
OK here is another (may be better) design.
Here is an example of the cfg script:
{
eth
{
dst: eth_addr(AA:BB:CC:DD:EE:FF)
proto: ipv4,
}
ip
{
src: ip_addr(192.168.1.1)
dst: ip_addr(192.168.1.2)
}
tcp
{
flags: syn | ack
}
}
Needs some refactoring of trafgen code existing cfg funcs which allows to use
manipulate of
current packet offset variable.
Each proto should be registered via struct proto_gen:
struct proto_field {
int id;
uint32_t offset;
uint32_t len;
};
struct proto_value {
void *value;
uint32_t len;
};
struct proto_gen {
struct cfg_ctx *ctx;
/* get field name info - id, len, offset */
struct proto_field *field_parse(struct proto_gen *prot, char *name);
/* indicates that this field was filled */
void field_set(struct proto_gen *proto, struct proto_field *field);
/* get info about some proto defined const value */
struct proto_value *value_get(struct proto_gen *proto, struct proto_field
*field, char *name);
/* fill the rest unfilled fields */
void fill(struct proto_gen *prot, uint8_t *hdr);
/* finish after whole packet was built */
void finish(struct proto_gen *prot, uint8_t *data, int len);
}
Each proto hdr field can be filled by specifying field name separated by ':' and
macro/func which will fill the bytes.The field may be filled with any
existing trafgen macro (const(), rnd() - the field len should be
considered). Also some additional macro/funcs should be added - ipv4_addr,
eth_addr, etc.
The parser will lookup keep struct proto_gen proto_ptr when the proto section
was entered (when '#proto {' was parsed), and parser can call the
proto_ptr-field_parse to obtain the field information by name (id,
offset, len) then change the current packet offset call the macro/func
which will fill the field. Then parser should call proto_ptr-field_set
to let know to the proto_gen which field was filled. At the end parser
will call proto_ptr-fill to fill the rest unfilled fields with some
default values fill the csum fields if needed.
While filling the default fields values the proto_gen should know some
trafgen's context info - interface id, and may be some other info which may
help to construct such unfilled fields like dst/src Ethernet addresses or
src/dst IP addresses.
Also it is possible that each proto_gen may have some own defined values like
flag names which can be evaluated via proto_gen-value_get where field pointer
may be specified. It might be useful if these values will be a part of
expressions
like: syn | ack.
At the end the parser will walk over each proto from higher layer to do a last
call
proto_gen-finish where each proto may do some calculations and fill some
unfilled fields.
Meanwhile I am not sure if it will be needed.
Well this is very conceptually and I will try to prepare better version which
would consider the existing trafgen code cfg syntax parser.
Ok, how would all of this tie together with flex and bison that we
use for trafgen as parser generator? It looks on a first glance that
this design would be outside of that scope?
Cheers,
Daniel
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On Fri, Jun 19, 2015 at 12:08:57AM +0200, Daniel Borkmann wrote:
On 06/18/2015 03:24 PM, Vadim Kochan wrote:
On Thu, Jun 18, 2015 at 11:07:12AM +0200, Daniel Borkmann wrote:
On 06/18/2015 10:57 AM, Vadim Kochan wrote:
...
Not yet, but I will try keep thinking about this, and will let know if I
will have some real ideas regarding implementation.
Sounds good!
Some 1st though which I came up is may be to start extend current
trafgen generation api set:
{
eth
{
src(),
dst(),
proto(),
}
,
ip
{
ver(),
src(),
dst(),
opt(0x1, 20),
opt(0x2, 30),
},
tcp
{
dport(),
sport(),
flags(),
},
}
Can you elaborate on your idea, what would this config do? I.e. if src()
etc is empty.
Thanks,
Daniel
Hi,
OK here is another (may be better) design.
Here is an example of the cfg script:
{
eth
{
dst: eth_addr(AA:BB:CC:DD:EE:FF)
proto: ipv4,
}
ip
{
src: ip_addr(192.168.1.1)
dst: ip_addr(192.168.1.2)
}
tcp
{
flags: syn | ack
}
}
Needs some refactoring of trafgen code existing cfg funcs which allows to use
manipulate of
current packet offset variable.
Each proto should be registered via struct proto_gen:
struct proto_field {
int id;
uint32_t offset;
uint32_t len;
};
struct proto_value {
void *value;
uint32_t len;
};
struct proto_gen {
struct cfg_ctx *ctx;
/* get field name info - id, len, offset */
struct proto_field *field_parse(struct proto_gen *prot, char *name);
/* indicates that this field was filled */
void field_set(struct proto_gen *proto, struct proto_field *field);
/* get info about some proto defined const value */
struct proto_value *value_get(struct proto_gen *proto, struct proto_field
*field, char *name);
/* fill the rest unfilled fields */
void fill(struct proto_gen *prot, uint8_t *hdr);
/* finish after whole packet was built */
void finish(struct proto_gen *prot, uint8_t *data, int len);
}
Each proto hdr field can be filled by specifying field name separated by ':' and
macro/func which will fill the bytes.The field may be filled with any
existing trafgen macro (const(), rnd() - the field len should be
considered). Also some additional macro/funcs should be added - ipv4_addr,
eth_addr, etc.
The parser will lookup keep struct proto_gen proto_ptr when the proto section
was entered (when '#proto {' was parsed), and parser can call the
proto_ptr-field_parse to obtain the field information by name (id,
offset, len) then change the current packet offset call the macro/func
which will fill the field. Then parser should call proto_ptr-field_set
to let know to the proto_gen which field was filled. At the end parser
will call proto_ptr-fill to fill the rest unfilled fields with some
default values fill the csum fields if needed.
While filling the default fields values the proto_gen should know some
trafgen's context info - interface id, and may be some other info which may
help to construct such unfilled fields like dst/src Ethernet addresses or
src/dst IP addresses.
Also it is possible that each proto_gen may have some own defined values like
flag names which can be evaluated via proto_gen-value_get where field pointer
may be specified. It might be useful if these values will be a part of
expressions
like: syn | ack.
At the end the parser will walk over each proto from higher layer to do a last
call
proto_gen-finish where each proto may do some calculations and fill some
unfilled fields.
Meanwhile I am not sure if it will be needed.
Well this is very conceptually and I will try to prepare better version which
would consider the existing trafgen code cfg syntax parser.
Regards,
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On Thu, Jun 18, 2015 at 11:07:12AM +0200, Daniel Borkmann wrote:
On 06/18/2015 10:57 AM, Vadim Kochan wrote:
...
Not yet, but I will try keep thinking about this, and will let know if I
will have some real ideas regarding implementation.
Sounds good!
Some 1st though which I came up is may be to start extend current
trafgen generation api set:
{
eth
{
src(),
dst(),
proto(),
}
,
ip
{
ver(),
src(),
dst(),
opt(0x1, 20),
opt(0x2, 30),
},
tcp
{
dport(),
sport(),
flags(),
},
}
and after use them when add cmd line options interface.
Regards,
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On 06/18/2015 03:24 PM, Vadim Kochan wrote:
On Thu, Jun 18, 2015 at 11:07:12AM +0200, Daniel Borkmann wrote:
On 06/18/2015 10:57 AM, Vadim Kochan wrote:
...
Not yet, but I will try keep thinking about this, and will let know if I
will have some real ideas regarding implementation.
Sounds good!
Some 1st though which I came up is may be to start extend current
trafgen generation api set:
{
eth
{
src(),
dst(),
proto(),
}
,
ip
{
ver(),
src(),
dst(),
opt(0x1, 20),
opt(0x2, 30),
},
tcp
{
dport(),
sport(),
flags(),
},
}
Can you elaborate on your idea, what would this config do? I.e. if src()
etc is empty.
Thanks,
Daniel
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On 06/18/2015 01:30 AM, Vadim Kochan wrote: Hi, I tried to look over mz tool and try to start adding TX fast path as it is done in trafgen but I am not sure it will be easy, for me it seems the code is not easy to change. I have no some objective arguments but I feel that may be it would be good to start something like pktgen-ng or start writing mz from empty page ? What do you think ? I think in general, it would be useful to 'merge' mz into trafgen. No doubt it's quite some work, but that would allow us to have a clean, fresh implementation at some point. By 'merge', I mean to have mostly compatible syntax/features, but the underlying implementation would be from scratch. I.e. to let trafgen have an interactive mode, so you can have it as a shell on an appliance, and for quick tests, the direct cmdline mode from mz. Do you have any plans? Thanks, Daniel -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On Thu, Jun 18, 2015 at 10:51:44AM +0200, Daniel Borkmann wrote: On 06/18/2015 01:30 AM, Vadim Kochan wrote: Hi, I tried to look over mz tool and try to start adding TX fast path as it is done in trafgen but I am not sure it will be easy, for me it seems the code is not easy to change. I have no some objective arguments but I feel that may be it would be good to start something like pktgen-ng or start writing mz from empty page ? What do you think ? I think in general, it would be useful to 'merge' mz into trafgen. No doubt it's quite some work, but that would allow us to have a clean, fresh implementation at some point. By 'merge', I mean to have mostly compatible syntax/features, but the underlying implementation would be from scratch. I.e. to let trafgen have an interactive mode, so you can have it as a shell on an appliance, and for quick tests, the direct cmdline mode from mz. Do you have any plans? Not yet, but I will try keep thinking about this, and will let know if I will have some real ideas regarding implementation. Thanks, Daniel -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
On 06/18/2015 10:57 AM, Vadim Kochan wrote: ... Not yet, but I will try keep thinking about this, and will let know if I will have some real ideas regarding implementation. Sounds good! -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?
Hi, I tried to look over mz tool and try to start adding TX fast path as it is done in trafgen but I am not sure it will be easy, for me it seems the code is not easy to change. I have no some objective arguments but I feel that may be it would be good to start something like pktgen-ng or start writing mz from empty page ? What do you think ? Regards, Vadim Kochan -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
