While troubleshooting or developing new features in OVS, a considerable amount of time is spent analyzing flows (whether that's Openflow flows or datapath flows). Currently, OVS has tools to dump flows with different levels of verbosity as well as to filter flows prior to dumping them, e.g: 'ovs-ofctl dump-flows', 'ovs-appctl dpctl/dump-flows', etc.
The output of these commands is considered stable so it should be possible to write more layered tools that enable advanced flow analysis. However, the way flows are formatted into strings is not trivial to parse. This series proposes the introduction of a flow parsing library capable of parsing both Openflow and DPIF flows. The library is based on generic key-value and list parsers and a number of common decoders. Based on that, an Openflow flow parser and a DPIF flow parser are introduced by defining a way to decode each possible field and action they might contain. The library has the following features: - Parsed key-value pairs keep some metadata that refer to the original strings they were extracted from. That way the flows can be printed and formatted in flexible ways. - It includes a basic flow filtering mechanism. A filter can be defined combining logical (||, &&, !), arithmetical (<, >, =) or mask (~=) operations - It supports IPAddress and Ethernet masking (based on netaddr) - The decoder to use for each key (match or action) is set explicitly to avoid expensive runtime type-guessing. - The decoders to use for Openflow fields is automatically generated based on meta-flow.h - Additional dependencies: - netaddr: For IP and Ethernet Address management - pyparsing: For filtering syntax As a proof of concept of how this library might be used, the series' last patch includes a user program, called "ofparse" that prints openflow and datapath flows in different formats including colorizing the output (using "rich" [3] library), json format and a "logic" representation of the flow tables. The idea behind the logical view of the flow tables comes from Flavio Leitner <[email protected]>. My original thought was to only add the utility as a proof of usage in the RFC, not as part of the final patch. However, if having such tool in the ovs tree is considered useful, I can clean up the code, split it in smaller commits and post it in a separate patch set. More info on the tool is available via 'ofparse --help' and on the online documentation [4]. ofparse usage ------------- $ make $ cd python; python -m venv venv; . ./venv/bin/activate; pip install . $ ovs-ofctl dump-flows br-int | ofparse openflow logic $ ovs-dpctl dpctl/dump-flows -m | ofparse datapath logic Library usage ------------- >>> from ovs.flows.ofp import OFPFlow >>> flow = OFPFlow.from_string("cookie=0x2b32ab4d, table=41, n_packets=11, >>> n_bytes=462, priority=33,ip,reg15=0x2/0x2,nw_src=10.128.0.2/24 >>> actions=move:NXM_OF_TCP_DST[]->NXM_NX_XXREG0[32..47],ct(table=16,zone=NXM_NX_REG13[0..15],nat)") >>> flow.info {'cookie': 724740941, 'table': 41, 'n_packets': 11, 'n_bytes': 462} >>> flow.match {'priority': 33, 'ip': True, 'reg15': Mask32('0x2/0x2'), 'nw_src': IPMask('10.128.0.2/24')} >>> flow.actions [{'move': {'src': {'field': 'NXM_OF_TCP_DST'}, 'dst': {'field': 'NXM_NX_XXREG0', 'start': 32, 'end': 47}}}, {'ct': {'table': 16, 'zone': {'field': 'NXM_NX_REG13', 'start': 0, 'end': 15}, 'nat': True}}] >>> from ovs.flows.filter import OFFilter >>> filt = OFFilter("nw_src ~= 10.128.0.10 and (table = 42 or n_packets > 0)") >>> filt.evaluate(flow) True Apart from the overall idea and approach, I would like to get feedback from the community on a number of aspects I'm not very clear about: - Apart from the openflow fields information, is there any other part of the decoding logic that can be automatically generated from ovs headers to improve maintainability? - Is there interest in adding specific utilities (such as ofparse) to the ovs tree? - While writing the library, I've written some unit tests. What do you think about adding python unit-tests? TODO: - properly parse flags (e.g: ct_state, tcp_flags, etc) - documentation Adrian Moreno (11): python: add generic Key-Value parser python: add mask, ip and eth decoders python: add list parser build-aux: split extract-ofp-fields build-aux: generate ofp field decoders python: add flow base class python: introduce OpenFlow Flow parsing python: add ovs datapath flow parsing python: add flow filtering syntax python: add a json encoder to flow fields python: add ofparse utility build-aux/automake.mk | 3 +- build-aux/extract-ofp-fields | 393 +--------------- build-aux/gen_ofp_field_decoders | 73 +++ python/.gitignore | 1 + python/automake.mk | 31 +- python/build/extract_ofp_fields.py | 386 ++++++++++++++++ python/ovs/flows/__init__.py | 0 python/ovs/flows/decoders.py | 458 +++++++++++++++++++ python/ovs/flows/filter.py | 158 +++++++ python/ovs/flows/flow.py | 81 ++++ python/ovs/flows/kv.py | 272 +++++++++++ python/ovs/flows/list.py | 121 +++++ python/ovs/flows/odp.py | 698 +++++++++++++++++++++++++++++ python/ovs/flows/ofp.py | 575 ++++++++++++++++++++++++ python/ovs/ofparse/__init__.py | 1 + python/ovs/ofparse/console.py | 248 ++++++++++ python/ovs/ofparse/dp.py | 102 +++++ python/ovs/ofparse/main.py | 118 +++++ python/ovs/ofparse/ofp.py | 167 +++++++ python/ovs/ofparse/ofparse | 6 + python/ovs/ofparse/process.py | 82 ++++ python/setup.py | 5 +- 22 files changed, 3589 insertions(+), 390 deletions(-) create mode 100755 build-aux/gen_ofp_field_decoders create mode 100644 python/build/extract_ofp_fields.py create mode 100644 python/ovs/flows/__init__.py create mode 100644 python/ovs/flows/decoders.py create mode 100644 python/ovs/flows/filter.py create mode 100644 python/ovs/flows/flow.py create mode 100644 python/ovs/flows/kv.py create mode 100644 python/ovs/flows/list.py create mode 100644 python/ovs/flows/odp.py create mode 100644 python/ovs/flows/ofp.py create mode 100644 python/ovs/ofparse/__init__.py create mode 100644 python/ovs/ofparse/console.py create mode 100644 python/ovs/ofparse/dp.py create mode 100644 python/ovs/ofparse/main.py create mode 100644 python/ovs/ofparse/ofp.py create mode 100755 python/ovs/ofparse/ofparse create mode 100644 python/ovs/ofparse/process.py -- 2.31.1 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
